Re: [Python-Dev] Rough idea for adding introspection information for builtins
On Mon, Mar 18, 2013 at 11:08 PM, Stefan Behnel wrote: > I can't see why the size would matter in any way. We're mildly concerned about the possible impact on the size of the ever-growing CPython binaries. However, it turns out that this is a case where readability and brevity are allies rather than enemies, so we don't need to choose one or the other. > > >> Of those, Nick's suggestion seems best. It's slightly bigger than the >> specialized bytecode format, but it's human-readable (and human-writable!), >> and it'd be the easiest to implement. > > Plus, if it becomes the format how C level signatures are expressed anyway, > it wouldn't require any additional build time preprocessing. > > >> My first idea for implementation: add a "def x" to the front and ": pass" >> to the end > > Why not require it to be there already? Maybe more like > > def foo(arg, b=3, *, kwonly='a'): > ... > > (i.e. using Ellipsis instead of pass, so that it's clear that it's not an > empty function but one the implementation of which is hidden) I like this notion. The groups notation and '/' will still cause the parser to choke and require special handling, but OTOH, they have deliberately been chosen as potentially acceptable notations for providing the same features in actual Python function declarations. > > >> then run it through ast.parse. Iterate over the tree, >> converting parameters into inspect.Parameters and handling the return >> annotation if present. Default values and annotations would be turned into >> values by ast.eval_literal. (It wouldn't surprise me if there's a cleaner >> way to do it than the fake function definition; I'm not familiar with the >> ast module.) > > IMHO, if there is no straight forward way currently to convert a function > header from a code blob into a Signature object in Python code, preferably > using the ast module (either explicitly or implicitly through inspect.py), > then that's a bug. The complexity here is that Larry would like to limit the annotations to compatibility with ast.literal_eval. If we drop that restriction, then the inspect module could handle the task directly. Given the complexity of implementing it, I believe the restriction needs more justification than is currently included in the PEP. > > >> We'd want one more mild hack: the DSL will support positional parameters, >> and inspect.Signature supports positional parameters, so it'd be nice to >> render that information. But we can't represent that in Python syntax (or >> at least not yet!), so we can't let ast.parse see it. My suggestion: run >> it through ast.parse, and if it throws a SyntaxError see if the problem was >> a slash. If it was, remove the slash, reprocess through ast.parse, and >> remember that all parameters are positional-only (and barf if there are >> kwonly, args, or kwargs). > > Is sounds simpler to me to just make it a Python syntax feature. Or at > least an optional one, supported by the ast module with a dedicated > compiler flag. Agreed. Guido had previously decided "not worth the hassle", but this may be enough to make him change his mind. Also, Larry's "simple" solution here isn't enough, since it doesn't handle optional groups correctly. While the support still has some odd limitations under the covers, I think an explicit compiler flag is a good compromise between a lot of custom hacks and exposing an unfinished implementation of a new language feature. Cheers, Nick. > > Stefan > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rough idea for adding introspection information for builtins
On 19.03.13 06:45, Larry Hastings wrote: 4. Store a string that looks like the Python declaration of the signature, and parse it (Nick's suggestion). For foo above, this would be "(arg,b=3,*,kwonly='a')". Length: 23 bytes. Strip parenthesis and it will be only 21 bytes long. We'd want one more mild hack: the DSL will support positional parameters, and inspect.Signature supports positional parameters, so it'd be nice to render that information. But we can't represent that in Python syntax (or at least not yet!), so we can't let ast.parse see it. My suggestion: run it through ast.parse, and if it throws a SyntaxError see if the problem was a slash. If it was, remove the slash, reprocess through ast.parse, and remember that all parameters are positional-only (and barf if there are kwonly, args, or kwargs). It will be simpler to use some one-character separator which shouldn't be used unquoted in the signature. I.e. LF. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Tarfile CLI
There is a proposition to add a command line interface to tarfile module. It will be useful on platforms where tar is not included in the base system. The question is about interface. Should it be a subset of tar options (-t as --extract, -t as --list, -f to specify an archive name) or be similar to the zipfile module interface (-e as --extract, -l as --list, the first positional parameter is an archive name)? There are different opinions. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tarfile CLI
On 19.03.13 09:52, Serhiy Storchaka wrote: There is a proposition to add a command line interface to tarfile module. Link: http://bugs.python.org/issue13477 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Recent changes to TextIOWrapper and its tests
On 18.03.13 22:26, Jeff Allen wrote: The puzzle is that it requires t.read() to succeed. When I insert a check for bytes type in all the places it seems necessary in my code, I pass the first two conditions, but since t.read() also raises TypeError, the overall test fails. Is reading the stream with read() intended to succeed? Why is this desired? And alternative option is to change C implementation of TextIOWrapper.read() to raise an exception in this case. However I worry that it can break backward compatibility. Are there other tests (in other test files) which fail with a new Jython TextIOWrapper? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rough idea for adding introspection information for builtins
On 03/19/2013 12:37 AM, Serhiy Storchaka wrote: On 19.03.13 06:45, Larry Hastings wrote: 4. Store a string that looks like the Python declaration of the signature, and parse it (Nick's suggestion). For foo above, this would be "(arg,b=3,*,kwonly='a')". Length: 23 bytes. Strip parenthesis and it will be only 21 bytes long. I left the parentheses there because the return annotation is outside them. If we strip the parentheses, I would have to restore them, and if there was a return annotation I would have to parse the string to know where to put it, because there could be arbitrary Python rvalues on either side of it with quotes and everything, and now I can no longer use ast.parse because it's not legal Python because the parentheses are missing ;-) We could omit the /left/ parenthesis and save one byte per builtin. I honestly don't know how many builtins there are, but my guess is one extra byte per builtin isn't a big deal. Let's leave it in for readability's sakes. We'd want one more mild hack: the DSL will support positional parameters, and inspect.Signature supports positional parameters, so it'd be nice to render that information. But we can't represent that in Python syntax (or at least not yet!), so we can't let ast.parse see it. My suggestion: run it through ast.parse, and if it throws a SyntaxError see if the problem was a slash. If it was, remove the slash, reprocess through ast.parse, and remember that all parameters are positional-only (and barf if there are kwonly, args, or kwargs). It will be simpler to use some one-character separator which shouldn't be used unquoted in the signature. I.e. LF. I had trouble understanding what you're suggesting. What I think you're saying is, "normally these generated strings won't have LF in them. So let's use LF as a harmless extra character that means 'this is a positional-only signature'." At one point Guido suggested / as syntax for exactly this case. And while the LF approach is simpler programmatically, removing the slash and reparsing isn't terribly complicated; this part will be in Python, after all. Meanwhile, I suggest that for human readability the slash is way more obvious--having a LF in the string mean this is awfully subtle. //arry/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rough idea for adding introspection information for builtins
On 19 Mar, 2013, at 10:24, Larry Hastings wrote: > > >>> We'd want one more mild hack: the DSL will support positional >>> parameters, and inspect.Signature supports positional parameters, so >>> it'd be nice to render that information. But we can't represent that in >>> Python syntax (or at least not yet!), so we can't let ast.parse see it. >>> My suggestion: run it through ast.parse, and if it throws a SyntaxError >>> see if the problem was a slash. If it was, remove the slash, reprocess >>> through ast.parse, and remember that all parameters are positional-only >>> (and barf if there are kwonly, args, or kwargs). >> >> It will be simpler to use some one-character separator which shouldn't be >> used unquoted in the signature. I.e. LF. > > I had trouble understanding what you're suggesting. What I think you're > saying is, "normally these generated strings won't have LF in them. So let's > use LF as a harmless extra character that means 'this is a positional-only > signature'." > > At one point Guido suggested / as syntax for exactly this case. And while > the LF approach is simpler programmatically, removing the slash and reparsing > isn't terribly complicated; this part will be in Python, after all. > Meanwhile, I suggest that for human readability the slash is way more > obvious--having a LF in the string mean this is awfully subtle. You could also add the slash to the start of the signature, for example "/(arg1, arg2)", that way the positional only can be detected without trying to parse it first and removing a slash at the start is easier than removing it somewhere along a signature with arbitrary default values, such as "(arg1='/', arg2=4 /) -> 'arg1/arg2'". The disadvantage is that you can't specify that only some of the arguments are positional-only, but that's not supported by PyArg_Parse... anyway. Ronald ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rough idea for adding introspection information for builtins
On 03/19/2013 12:23 AM, Nick Coghlan wrote: On Mon, Mar 18, 2013 at 11:08 PM, Stefan Behnel wrote: My first idea for implementation: add a "def x" to the front and ": pass" to the end Why not require it to be there already? Maybe more like def foo(arg, b=3, *, kwonly='a'): ... (i.e. using Ellipsis instead of pass, so that it's clear that it's not an empty function but one the implementation of which is hidden) I like this notion. The groups notation and '/' will still cause the parser to choke and require special handling, but OTOH, they have deliberately been chosen as potentially acceptable notations for providing the same features in actual Python function declarations. I don't see the benefit of including the "def foo" and ":\n ...". The name doesn't help; inspect.Signature pointedly does /not/ contain the name of the function, so it's irrelevant to this purpose. And why have unnecessary boilerplate? And if I can go one further: what we're talking about is essentially a textual representation of a Signature object. I assert that the stuff inside the parentheses, and the return annotation, *is* the signature. The name isn't part of the signature, and the colon and what lies afterwards is definitely not part of its signature. So I think it's entirely appropriate, and a happy coincidence, that it happens to reflect the minimum amount of text you need to communicate the signature. IMHO, if there is no straight forward way currently to convert a function header from a code blob into a Signature object in Python code, preferably using the ast module (either explicitly or implicitly through inspect.py), then that's a bug. The complexity here is that Larry would like to limit the annotations to compatibility with ast.literal_eval. If we drop that restriction, then the inspect module could handle the task directly. Given the complexity of implementing it, I believe the restriction needs more justification than is currently included in the PEP. I concede that it's totally unjustified in the PEP. It's more playing a hunch at the moment, a combination of YAGNI and that it'd be hard to put the genie back in the bottle if we let people use arbitrary values. Let me restate what we're talking about. We're debating what types of data should be permissible to use for a datum that so far is not only unused, but is /required/ to be unused. PEP 8 states " The Python standard library will not use function annotations". I don't know who among us has any experience using function annotations--or, at least, for their intended purpose. It's hard to debate what are reasonable vs unreasonable restrictions on data we might be permitted to specify in the future for uses we don't know about. Restricting it to Python's rich set of safe literal values seems entirely reasonable; if we get there and need to relax the restriction, we can do so there. Also, you and I discussed this evening whether there was a credible attack vector here. I figured, if you're running an untrustworthy extension, it's already game over. You suggested that a miscreant could easily edit static data on a trusted shared library without having to recompile it to achieve their naughtiness. I'm not sure I necessarily buy it, I just wanted to point out you were the one making the case for restricting it to ast.literal_eval. ;-) Is sounds simpler to me to just make it a Python syntax feature. Or at least an optional one, supported by the ast module with a dedicated compiler flag. Agreed. Guido had previously decided "not worth the hassle", but this may be enough to make him change his mind. Also, Larry's "simple" solution here isn't enough, since it doesn't handle optional groups correctly. I certainly don't agree that "remove the slash and reparse" is more complicated than "add a new parameter metaphor to the Python language". Adding support for it may be worth doing--don't ask me, I'm still nursing my "positional-only arguments are part of Python and forever will be" Kool-aid. I'm just dealing with cold harsh reality as I understand it. As for handling optional argument groups, my gut feeling is that we're better off not leaking it out of Argument Clinic--don't expose it in this string we're talking about, and don't add support for it in the inspect.Parameter object. I'm not going to debate range(), the syntax of which predates one of our release managers. But I suggest option groups are simply a misfeature of the curses module. There are some other possible uses in builtins (I forgot to dig those out this evening) but so far we're talking adding complexity to an array of technologies (this representation, the parser, the Parameter object) to support a handful of uses of something we shouldn't have done in the first place, for consumers who I think won't care and won't appreciate the added conceptual complexity. //arry/ __
Re: [Python-Dev] Rough idea for adding introspection information for builtins
Le Tue, 19 Mar 2013 03:00:45 -0700, Larry Hastings a écrit : > > As for handling optional argument groups, my gut feeling is that > we're better off not leaking it out of Argument Clinic--don't expose > it in this string we're talking about, and don't add support for it > in the inspect.Parameter object. I'm not going to debate range(), > the syntax of which predates one of our release managers. But I > suggest option groups are simply a misfeature of the curses module. > There are some other possible uses in builtins (I forgot to dig those > out this evening) but so far we're talking adding complexity to an > array of technologies (this representation, the parser, the Parameter > object) to support a handful of uses of something we shouldn't have > done in the first place, for consumers who I think won't care and > won't appreciate the added conceptual complexity. Agreed with Larry. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rough idea for adding introspection information for builtins
On Tue, Mar 19, 2013 at 3:00 AM, Larry Hastings wrote:
> Why not require it to be there already? Maybe more like
>
> def foo(arg, b=3, *, kwonly='a'):
> ...
>
> (i.e. using Ellipsis instead of pass, so that it's clear that it's not an
> empty function but one the implementation of which is hidden)
>
> I like this notion. The groups notation and '/' will still cause the
> parser to choke and require special handling, but OTOH, they have
> deliberately been chosen as potentially acceptable notations for
> providing the same features in actual Python function declarations.
>
>
> I don't see the benefit of including the "def foo" and ":\n...". The
> name doesn't help; inspect.Signature pointedly does not contain the name of
> the function, so it's irrelevant to this purpose. And why have unnecessary
> boilerplate?
Also, we can already easily produce the extended form through:
"def {}{}:\n...".format(f.__name__, inspect.signature(f))
So, agreed, capturing just the signature info is fine.
> Let me restate what we're talking about. We're debating what types of data
> should be permissible to use for a datum that so far is not only unused, but
> is required to be unused. PEP 8 states " The Python standard library will
> not use function annotations". I don't know who among us has any experience
> using function annotations--or, at least, for their intended purpose. It's
> hard to debate what are reasonable vs unreasonable restrictions on data we
> might be permitted to specify in the future for uses we don't know about.
> Restricting it to Python's rich set of safe literal values seems entirely
> reasonable; if we get there and need to relax the restriction, we can do so
> there.
>
> Also, you and I discussed this evening whether there was a credible attack
> vector here. I figured, if you're running an untrustworthy extension, it's
> already game over. You suggested that a miscreant could easily edit static
> data on a trusted shared library without having to recompile it to achieve
> their naughtiness. I'm not sure I necessarily buy it, I just wanted to
> point out you were the one making the case for restricting it to
> ast.literal_eval. ;-)
IIRC, I was arguing against allowing *pickle* because you can't audit
that just by looking at the generated source code. OTOH, I'm a big fan
of locking this kind of thing down by default and letting people make
the case for additional permissiveness, so I agree it's best to start
with literals only.
Here's a thought, though: instead of doing an Argument Clinic specific
hack, let's instead design a proper whitelist API for ast.literal_eval
that lets you accept additional constructs.
As a general sketch, the long if/elif chain in ast.literal_eval could
be replaced by:
for converter in converters:
ok, converted = converter(node)
if ok:
return converted
raise ValueError('malformed node or string: ' + repr(node))
The _convert function would need to be lifted out and made public as
"ast.convert_node", so conversion functions could recurse
appropriately.
Both ast.literal_eval and ast.convert_node would accept a keyword-only
"allow" parameter that accepted an iterable of callables that return a
2-tuple to whitelist additional expressions beyond those normally
allowed. So, assuming we don't add it by default, you could allow
empty sets by doing:
_empty_set = ast.dump(ast.parse("set()").body[0].value)
def convert_empty_set(node):
if ast.dump(node) == _empty_set:
return True, set()
return False, None
ast.literal_eval(some_str, allow=[convert_empy_set])
This is quite powerful as a general tool to allow constrained
execution, since it could be used to whitelist builtins that accept
parameters, as well as to process class and function header lines
without executing their bodies. In the case of Argument Clinic, that
would mean writing a converter for the FunctionDef node.
> I certainly don't agree that "remove the slash and reparse" is more
> complicated than "add a new parameter metaphor to the Python language".
> Adding support for it may be worth doing--don't ask me, I'm still nursing my
> "positional-only arguments are part of Python and forever will be" Kool-aid.
> I'm just dealing with cold harsh reality as I understand it.
>
> As for handling optional argument groups, my gut feeling is that we're
> better off not leaking it out of Argument Clinic--don't expose it in this
> string we're talking about, and don't add support for it in the
> inspect.Parameter object. I'm not going to debate range(), the syntax of
> which predates one of our release managers. But I suggest option groups are
> simply a misfeature of the curses module. There are some other possible
> uses in builtins (I forgot to dig those out this evening) but so far we're
> talking adding complexity to an array of technologies (this representation,
> the parser, the Parameter object) to suppor
Re: [Python-Dev] [Python-checkins] peps: New DSL syntax and slightly changed semantics for the Argument Clinic DSL.
Guido van Rossum wrote: > On Mon, Mar 18, 2013 at 3:36 AM, Stefan Krah wrote: > > Larry Hastings wrote: > >> * The DSL currently makes no provision for specifying per-parameter > >> type annotations. This is something explicitly supported in Python; > >> it should be supported for builtins too, once we have reflection support. > >> > >> It seems to me that the syntax for parameter lines--dictated by > >> Guido--suggests conversion functions are themselves type annotations. > >> This makes intuitive sense. > > > > Really, did you read PEP 437? It's all in there. > > This attitude is unhelpful. Please stop being outright hostile. If you > want to have any influence on the outcome at all, consider looking > into compromises. My apologies, I agree that wasn't very constructive. In case there's a misunderstanding: This wasn't an attempt to push the whole of PEP 437 again. Type-specifying converters were first mentioned in issue #16612 and are central to PEP 437, so my response should have been something like "I think that's already covered in section X of ...". Regarding compromises: I'm not at all after getting as many parts of PEP 437 into the end result as possible. Apparently PEP 437 as a whole is unacceptable, so if Larry's original PEP 436 turns out to be more coherent than the revised PEP 436, then I'd actually favor Larry's original. I'm getting the impression though that the proposal that Guido, Larry and Nick worked out at PyCon *is* reasonably coherent. So my position is the same as Nick's in http://mail.python.org/pipermail/python-dev/2013-March/124757.html Stefan Krah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rough idea for adding introspection information for builtins
On Mar 18, 2013, at 09:45 PM, Larry Hastings wrote: >4. Store a string that looks like the Python declaration of the signature, >and parse it (Nick's suggestion). For foo above, this would be >"(arg,b=3,*,kwonly='a')". Length: 23 bytes. Very nice. -Barry ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: ../bug-fixes/http_error_interface/.hg/last-message.txt
Looks like I used hg commit -m /path/to/.hg/last-message.txt instead
of hg commit -l /path/to/.hg/last-message.txt
I have amended it, merged it and pushed it again.
On Tue, Mar 19, 2013 at 12:04 PM, senthil.kumaran
wrote:
> http://hg.python.org/cpython/rev/4f2080e9eee2
> changeset: 82765:4f2080e9eee2
> parent: 82763:4c6463b96a2c
> user:Senthil Kumaran
> date:Tue Mar 19 12:07:43 2013 -0700
> summary:
> ../bug-fixes/http_error_interface/.hg/last-message.txt
>
> files:
> Lib/test/test_urllib2.py | 39 +--
> 1 files changed, 19 insertions(+), 20 deletions(-)
>
>
> diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py
> --- a/Lib/test/test_urllib2.py
> +++ b/Lib/test/test_urllib2.py
> @@ -1387,6 +1387,10 @@
>
> class MiscTests(unittest.TestCase):
>
> +def opener_has_handler(self, opener, handler_class):
> +self.assertTrue(any(h.__class__ == handler_class
> +for h in opener.handlers))
> +
> def test_build_opener(self):
> class MyHTTPHandler(urllib.request.HTTPHandler): pass
> class FooHandler(urllib.request.BaseHandler):
> @@ -1439,10 +1443,22 @@
> self.assertEqual(b"1234567890", request.data)
> self.assertEqual("10", request.get_header("Content-length"))
>
> +def test_HTTPError_interface(self):
> +"""
> +Issue 13211 reveals that HTTPError didn't implement the URLError
> +interface even though HTTPError is a subclass of URLError.
>
> -def opener_has_handler(self, opener, handler_class):
> -self.assertTrue(any(h.__class__ == handler_class
> -for h in opener.handlers))
> +>>> msg = 'something bad happened'
> +>>> url = code = fp = None
> +>>> hdrs = 'Content-Length: 42'
> +>>> err = urllib.error.HTTPError(url, code, msg, hdrs, fp)
> +>>> assert hasattr(err, 'reason')
> +>>> err.reason
> +'something bad happened'
> +>>> assert hasattr(err, 'headers')
> +>>> err.headers
> +'Content-Length: 42'
> +"""
>
> class RequestTests(unittest.TestCase):
>
> @@ -1514,23 +1530,6 @@
> req = Request(url)
> self.assertEqual(req.get_full_url(), url)
>
> -def test_HTTPError_interface():
> -"""
> -Issue 13211 reveals that HTTPError didn't implement the URLError
> -interface even though HTTPError is a subclass of URLError.
> -
> ->>> msg = 'something bad happened'
> ->>> url = code = fp = None
> ->>> hdrs = 'Content-Length: 42'
> ->>> err = urllib.error.HTTPError(url, code, msg, hdrs, fp)
> ->>> assert hasattr(err, 'reason')
> ->>> err.reason
> -'something bad happened'
> ->>> assert hasattr(err, 'headers')
> ->>> err.headers
> -'Content-Length: 42'
> -"""
> -
> def test_main(verbose=None):
> from test import test_urllib2
> support.run_doctest(test_urllib2, verbose)
>
> --
> Repository URL: http://hg.python.org/cpython
>
> ___
> Python-checkins mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-checkins
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout
Yes, it is a symbol problem on unix. Working on it. -Original Message- From: Python-checkins [mailto:[email protected]] On Behalf Of Senthil Kumaran Sent: 19. mars 2013 12:28 To: [email protected] Cc: [email protected] Subject: Re: [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout Looks like RHEL 2.7 buildbots are unhappy with this change. -- Senthil On Tue, Mar 19, 2013 at 11:08 AM, kristjan.jonsson wrote: > http://hg.python.org/cpython/rev/8ec39bfd1f01 > changeset: 82764:8ec39bfd1f01 > branch: 2.7 > parent: 82740:b10ec5083a53 > user:Kristján Valur Jónsson > date:Tue Mar 19 10:58:59 2013 -0700 > summary: > Issue #9090 : Error code 10035 calling socket.recv() on a socket > with a timeout (WSAEWOULDBLOCK - A non-blocking socket operation > could not be completed > immediately) > > files: > Misc/NEWS |5 + > Modules/socketmodule.c | 104 > Modules/timemodule.c |7 + > 3 files changed, 101 insertions(+), 15 deletions(-) > > > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -214,6 +214,11 @@ > Library > --- > > +- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or > +EAGAIN, > + retry the select() loop instead of bailing out. This is because > +select() > + can incorrectly report a socket as ready for reading (for example, > +if it > + received some data with an invalid checksum). > + > - Issue #1285086: Get rid of the refcounting hack and speed up > urllib.unquote(). > > - Issue #17368: Fix an off-by-one error in the Python JSON decoder > that caused diff --git a/Modules/socketmodule.c > b/Modules/socketmodule.c > --- a/Modules/socketmodule.c > +++ b/Modules/socketmodule.c > @@ -473,6 +473,17 @@ > return NULL; > } > > +#ifdef MS_WINDOWS > +#ifndef WSAEAGAIN > +#define WSAEAGAIN WSAEWOULDBLOCK > +#endif > +#define CHECK_ERRNO(expected) \ > +(WSAGetLastError() == WSA ## expected) #else #define > +CHECK_ERRNO(expected) \ > +(errno == expected) > +#endif > + > /* Convenience function to raise an error according to errno > and return a NULL pointer from a function. */ > > @@ -661,7 +672,7 @@ > after they've reacquired the interpreter lock. > Returns 1 on timeout, -1 on error, 0 otherwise. */ static int > -internal_select(PySocketSockObject *s, int writing) > +internal_select_ex(PySocketSockObject *s, int writing, double > +interval) > { > int n; > > @@ -673,6 +684,10 @@ > if (s->sock_fd < 0) > return 0; > > +/* Handling this condition here simplifies the select loops */ > +if (interval < 0.0) > +return 1; > + > /* Prefer poll, if available, since you can poll() any fd > * which can't be done with select(). */ #ifdef HAVE_POLL @@ > -684,7 +699,7 @@ > pollfd.events = writing ? POLLOUT : POLLIN; > > /* s->sock_timeout is in seconds, timeout in ms */ > -timeout = (int)(s->sock_timeout * 1000 + 0.5); > +timeout = (int)(interval * 1000 + 0.5); > n = poll(&pollfd, 1, timeout); > } > #else > @@ -692,8 +707,8 @@ > /* Construct the arguments to select */ > fd_set fds; > struct timeval tv; > -tv.tv_sec = (int)s->sock_timeout; > -tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6); > +tv.tv_sec = (int)interval; > +tv.tv_usec = (int)((interval - tv.tv_sec) * 1e6); > FD_ZERO(&fds); > FD_SET(s->sock_fd, &fds); > > @@ -712,6 +727,49 @@ > return 0; > } > > +static int > +internal_select(PySocketSockObject *s, int writing) { > +return internal_select_ex(s, writing, s->sock_timeout); } > + > +/* > + Two macros for automatic retry of select() in case of false positives > + (for example, select() could indicate a socket is ready for reading > +but the data then discarded by the OS because of a wrong checksum). > + Here is an example of use: > + > +BEGIN_SELECT_LOOP(s) > +Py_BEGIN_ALLOW_THREADS > +timeout = internal_select_ex(s, 0, interval); > +if (!timeout) > +outlen = recv(s->sock_fd, cbuf, len, flags); > +Py_END_ALLOW_THREADS > +if (timeout == 1) { > +PyErr_SetString(socket_timeout, "timed out"); > +return -1; > +} > +END_SELECT_LOOP(s) > +*/ > +PyAPI_FUNC(double) _PyTime_floattime(void); /* defined in timemodule.c */ > +#define BEGIN_SELECT_LOOP(s) \ > +{ \ > +double deadline, interval = s->sock_timeout; \ > +int has_timeout = s->sock_timeout > 0.0; \ > +if (has_timeout) { \ > +deadline = _PyTime_floattime() + s->sock_timeout; \ > +} \ > +while (1) { \ > +errno = 0; \ > + > +#define END_SELECT_LOOP(s) \ > +if (!has_timeout || \ > +(!CHECK_ERRNO(EWOULDBLOCK) && !CHECK_ERRNO(EA
Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout
Apparently timemodule is not a built-in module on linux. But it is on windows. Funny! -Original Message- From: Python-Dev [mailto:[email protected]] On Behalf Of Kristján Valur Jónsson Sent: 19. mars 2013 12:34 To: [email protected] Subject: Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout Yes, it is a symbol problem on unix. Working on it. -Original Message- From: Python-checkins [mailto:[email protected]] On Behalf Of Senthil Kumaran Sent: 19. mars 2013 12:28 To: [email protected] Cc: [email protected] Subject: Re: [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout Looks like RHEL 2.7 buildbots are unhappy with this change. -- Senthil On Tue, Mar 19, 2013 at 11:08 AM, kristjan.jonsson wrote: > http://hg.python.org/cpython/rev/8ec39bfd1f01 > changeset: 82764:8ec39bfd1f01 > branch: 2.7 > parent: 82740:b10ec5083a53 > user:Kristján Valur Jónsson > date:Tue Mar 19 10:58:59 2013 -0700 > summary: > Issue #9090 : Error code 10035 calling socket.recv() on a socket > with a timeout (WSAEWOULDBLOCK - A non-blocking socket operation > could not be completed > immediately) > > files: > Misc/NEWS |5 + > Modules/socketmodule.c | 104 > Modules/timemodule.c |7 + > 3 files changed, 101 insertions(+), 15 deletions(-) > > > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -214,6 +214,11 @@ > Library > --- > > +- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or > +EAGAIN, > + retry the select() loop instead of bailing out. This is because > +select() > + can incorrectly report a socket as ready for reading (for example, > +if it > + received some data with an invalid checksum). > + > - Issue #1285086: Get rid of the refcounting hack and speed up > urllib.unquote(). > > - Issue #17368: Fix an off-by-one error in the Python JSON decoder > that caused diff --git a/Modules/socketmodule.c > b/Modules/socketmodule.c > --- a/Modules/socketmodule.c > +++ b/Modules/socketmodule.c > @@ -473,6 +473,17 @@ > return NULL; > } > > +#ifdef MS_WINDOWS > +#ifndef WSAEAGAIN > +#define WSAEAGAIN WSAEWOULDBLOCK > +#endif > +#define CHECK_ERRNO(expected) \ > +(WSAGetLastError() == WSA ## expected) #else #define > +CHECK_ERRNO(expected) \ > +(errno == expected) > +#endif > + > /* Convenience function to raise an error according to errno > and return a NULL pointer from a function. */ > > @@ -661,7 +672,7 @@ > after they've reacquired the interpreter lock. > Returns 1 on timeout, -1 on error, 0 otherwise. */ static int > -internal_select(PySocketSockObject *s, int writing) > +internal_select_ex(PySocketSockObject *s, int writing, double > +interval) > { > int n; > > @@ -673,6 +684,10 @@ > if (s->sock_fd < 0) > return 0; > > +/* Handling this condition here simplifies the select loops */ > +if (interval < 0.0) > +return 1; > + > /* Prefer poll, if available, since you can poll() any fd > * which can't be done with select(). */ #ifdef HAVE_POLL @@ > -684,7 +699,7 @@ > pollfd.events = writing ? POLLOUT : POLLIN; > > /* s->sock_timeout is in seconds, timeout in ms */ > -timeout = (int)(s->sock_timeout * 1000 + 0.5); > +timeout = (int)(interval * 1000 + 0.5); > n = poll(&pollfd, 1, timeout); > } > #else > @@ -692,8 +707,8 @@ > /* Construct the arguments to select */ > fd_set fds; > struct timeval tv; > -tv.tv_sec = (int)s->sock_timeout; > -tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6); > +tv.tv_sec = (int)interval; > +tv.tv_usec = (int)((interval - tv.tv_sec) * 1e6); > FD_ZERO(&fds); > FD_SET(s->sock_fd, &fds); > > @@ -712,6 +727,49 @@ > return 0; > } > > +static int > +internal_select(PySocketSockObject *s, int writing) { > +return internal_select_ex(s, writing, s->sock_timeout); } > + > +/* > + Two macros for automatic retry of select() in case of false positives > + (for example, select() could indicate a socket is ready for reading > +but the data then discarded by the OS because of a wrong checksum). > + Here is an example of use: > + > +BEGIN_SELECT_LOOP(s) > +Py_BEGIN_ALLOW_THREADS > +timeout = internal_select_ex(s, 0, interval); > +if (!timeout) > +outlen = recv(s->sock_fd, cbuf, len, flags); > +Py_END_ALLOW_THREADS > +if (timeout == 1) { > +PyErr_SetString(socket_timeout, "timed out"); > +return -1; > +} > +END_SELECT_LOOP(s) > +*/ > +PyAPI_FUNC(double) _PyTime_floattime(void); /* defined in > +timemodule.c */ #define BEGIN_SELECT_LOOP(s) \ > +
Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout
On Tue, Mar 19, 2013 at 11:08 AM, kristjan.jonsson < [email protected]> wrote: > http://hg.python.org/cpython/rev/8ec39bfd1f01 > changeset: 82764:8ec39bfd1f01 > branch: 2.7 > parent: 82740:b10ec5083a53 > user:Kristján Valur Jónsson > date:Tue Mar 19 10:58:59 2013 -0700 > summary: > Issue #9090 : Error code 10035 calling socket.recv() on a socket with a > timeout > (WSAEWOULDBLOCK - A non-blocking socket operation could not be completed > immediately) > [...] > +- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or > EAGAIN, > + retry the select() loop instead of bailing out. This is because > select() > + can incorrectly report a socket as ready for reading (for example, if it > + received some data with an invalid checksum). > Might I recommend treating EINTR the same way? It has the same issue of popping up, rarely, when you least expect it, and messing with your code. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Early results from Argument Clinic automation discussion
Mark Shannon, Dmitry Jemerov (from PyCharm) and I sat down to talk about rearchitecting the Argument Clinic prototype to make it easily to interact with. We came up with the following. The DSL will now produce an intermediate representation. The output will consume this intermediate representation. The two defined outputs from the IR so far: 1. The inplace generated C code (what it has right now), and 2. Argument Clinic DSL code itself. The intermediate representation will be subclasses of inspect.Signature and inspect.Parameter that add the extra bits of information needed by Argument Clinic, as follows: class Function(inspect.Signature): name = 'function name' module = 'module name if any class = 'class name if any, can't actually call this class' docstring = 'function docstring' c_id = 'stub id to use for generated c functions' return_converter = SeeBelow() class Parameter(inspect.Parameter): docstring = 'per-parameter docstring' group = converter = ConverterFunction() Parameter.group is an integer, specifying which "option group" the parameter is in. This must be an integer for positional-only arguments, and None for other argument types. 0 indicates "required positional-only parameter". Left-optional groups get negative numbers, decreasing as they get further from the required parameters; right-optional get positive numbers, increasing as they get further from the required parameters. (Groups cannot nest, so a parameter cannot be in more than one group.) Function.return_converter was suggested by Mark. This is the inverse of the per-parameter converter function: it defines the return type of the impl function, and the conversion process to turn it into the PyObject * that gets returned to Python. And, like the converter functions, it will define the actual return annotation of the function (if any). Mark wants to write something that parses C code implementing builtins and produces the IR; he can then use that to write Argument Clinic DSL. This will make the conversion process go much more quickly. //arry/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to
On Wed, 20 Mar 2013 01:22:58 +0100 (CET) michael.foord wrote: > http://hg.python.org/cpython/rev/684b75600fa9 > changeset: 82811:684b75600fa9 > user:Michael Foord > date:Tue Mar 19 17:22:51 2013 -0700 > summary: > Closes issue 17467. Add readline and readlines support to > unittest.mock.mock_open Wasn't it possible to re-use an existing implementation (such as TextIOBase or StringIO) rather than re-write your own? (it's not even obvious your implementation is correct, BTW. How about universal newlines?) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Issue #17385: Fix quadratic behavior in threading.Condition
On Mon, Mar 11, 2013 at 3:14 AM, Ezio Melotti wrote: > Hi, > > On Mon, Mar 11, 2013 at 2:58 AM, raymond.hettinger > wrote: >> http://hg.python.org/cpython/rev/0f86b51f8f8b >> changeset: 82592:0f86b51f8f8b >> user:Raymond Hettinger >> date:Sun Mar 10 17:57:28 2013 -0700 >> summary: >> Issue #17385: Fix quadratic behavior in threading.Condition >> >> files: >> Lib/threading.py | 10 -- >> Misc/NEWS| 3 +++ >> 2 files changed, 11 insertions(+), 2 deletions(-) >> >> >> diff --git a/Lib/threading.py b/Lib/threading.py >> --- a/Lib/threading.py >> +++ b/Lib/threading.py >> @@ -10,6 +10,12 @@ >> from time import time as _time >> from traceback import format_exc as _format_exc >> from _weakrefset import WeakSet >> +try: >> +from _itertools import islice as _slice >> +from _collections import deque as _deque >> +except ImportError: >> +from itertools import islice as _islice >> +from collections import deque as _deque >> > > Shouldn't the one in the 'try' be _islice too? > Also I don't seem to have an _itertools module. Is this something used by the other VMs? > Best Regards, > Ezio Melotti ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: #15927: Fix cvs.reader parsing of escaped \r\n with quoting off.
The compiler complains about this line:
if (c == '\n' | c=='\r') {
Perhaps you wanted a Boolean operator?
-Original Message-
From: Python-checkins
[mailto:[email protected]] On Behalf Of
r.david.murray
Sent: 19. mars 2013 19:42
To: [email protected]
Subject: [Python-checkins] cpython: #15927: Fix cvs.reader parsing of escaped
\r\n with quoting off.
http://hg.python.org/cpython/rev/940748853712
changeset: 82815:940748853712
parent: 82811:684b75600fa9
user:R David Murray
date:Tue Mar 19 22:41:47 2013 -0400
summary:
#15927: Fix cvs.reader parsing of escaped \r\n with quoting off.
This fix means that such values are correctly roundtripped, since cvs.writer
already does the correct escaping.
Patch by Michael Johnson.
files:
Lib/test/test_csv.py | 9 +
Misc/ACKS| 1 +
Misc/NEWS| 3 +++
Modules/_csv.c | 13 -
4 files changed, 25 insertions(+), 1 deletions(-)
diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -308,6 +308,15 @@
for i, row in enumerate(csv.reader(fileobj)):
self.assertEqual(row, rows[i])
+def test_roundtrip_escaped_unquoted_newlines(self):
+with TemporaryFile("w+", newline='') as fileobj:
+writer = csv.writer(fileobj,quoting=csv.QUOTE_NONE,escapechar="\\")
+rows = [['a\nb','b'],['c','x\r\nd']]
+writer.writerows(rows)
+fileobj.seek(0)
+for i, row in
enumerate(csv.reader(fileobj,quoting=csv.QUOTE_NONE,escapechar="\\")):
+self.assertEqual(row,rows[i])
+
class TestDialectRegistry(unittest.TestCase):
def test_registry_badargs(self):
self.assertRaises(TypeError, csv.list_dialects, None) diff --git
a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -591,6 +591,7 @@
Fredrik Johansson
Gregory K. Johnson
Kent Johnson
+Michael Johnson
Simon Johnston
Matt Joiner
Thomas Jollans
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -289,6 +289,9 @@
Library
---
+- Issue #15927: CVS now correctly parses escaped newlines and carriage
+ when parsing with quoting turned off.
+
- Issue #17467: add readline and readlines support to mock_open in
unittest.mock.
diff --git a/Modules/_csv.c b/Modules/_csv.c
--- a/Modules/_csv.c
+++ b/Modules/_csv.c
@@ -51,7 +51,7 @@
typedef enum {
START_RECORD, START_FIELD, ESCAPED_CHAR, IN_FIELD,
IN_QUOTED_FIELD, ESCAPE_IN_QUOTED_FIELD, QUOTE_IN_QUOTED_FIELD,
-EAT_CRNL
+EAT_CRNL,AFTER_ESCAPED_CRNL
} ParserState;
typedef enum {
@@ -644,6 +644,12 @@
break;
case ESCAPED_CHAR:
+if (c == '\n' | c=='\r') {
+if (parse_add_char(self, c) < 0)
+return -1;
+self->state = AFTER_ESCAPED_CRNL;
+break;
+}
if (c == '\0')
c = '\n';
if (parse_add_char(self, c) < 0) @@ -651,6 +657,11 @@
self->state = IN_FIELD;
break;
+case AFTER_ESCAPED_CRNL:
+if (c == '\0')
+break;
+/*fallthru*/
+
case IN_FIELD:
/* in unquoted field */
if (c == '\n' || c == '\r' || c == '\0') {
--
Repository URL: http://hg.python.org/cpython
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: #15927: Fix cvs.reader parsing of escaped \r\n with quoting off.
On Wed, 20 Mar 2013 03:16:53 -,
=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?= wrote:
> The compiler complains about this line:
> if (c == '\n' | c=='\r') {
>
> Perhaps you wanted a Boolean operator?
Indeed, yes.
--David
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] peps: update 2.7.4 release dates
Are you sure about 2.7.4 2012-04-06? I mean 2012 year. On Tue, Mar 19, 2013 at 9:15 PM, benjamin.peterson wrote: > http://hg.python.org/peps/rev/ce17779c395c > changeset: 4810:ce17779c395c > user:Benjamin Peterson > date:Tue Mar 19 23:15:23 2013 -0500 > summary: > update 2.7.4 release dates > > files: > pep-0373.txt | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-) > > > diff --git a/pep-0373.txt b/pep-0373.txt > --- a/pep-0373.txt > +++ b/pep-0373.txt > @@ -56,8 +56,8 @@ > > Planned future release dates: > > -- 2.7.4rc1 2013-02-02 > -- 2.7.4 2012-02-16 > +- 2.7.4rc1 2013-03-23 > +- 2.7.4 2012-04-06 > > Dates of previous maintenance releases: > > > -- > Repository URL: http://hg.python.org/peps > > ___ > Python-checkins mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-checkins > -- Thanks, Andrew Svetlov ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] peps: update 2.7.4 release dates
Good catch. 2013/3/19 Andrew Svetlov : > Are you sure about 2.7.4 2012-04-06? I mean 2012 year. > > On Tue, Mar 19, 2013 at 9:15 PM, benjamin.peterson > wrote: >> http://hg.python.org/peps/rev/ce17779c395c >> changeset: 4810:ce17779c395c >> user:Benjamin Peterson >> date:Tue Mar 19 23:15:23 2013 -0500 >> summary: >> update 2.7.4 release dates >> >> files: >> pep-0373.txt | 4 ++-- >> 1 files changed, 2 insertions(+), 2 deletions(-) >> >> >> diff --git a/pep-0373.txt b/pep-0373.txt >> --- a/pep-0373.txt >> +++ b/pep-0373.txt >> @@ -56,8 +56,8 @@ >> >> Planned future release dates: >> >> -- 2.7.4rc1 2013-02-02 >> -- 2.7.4 2012-02-16 >> +- 2.7.4rc1 2013-03-23 >> +- 2.7.4 2012-04-06 >> >> Dates of previous maintenance releases: >> >> >> -- >> Repository URL: http://hg.python.org/peps >> >> ___ >> Python-checkins mailing list >> [email protected] >> http://mail.python.org/mailman/listinfo/python-checkins >> > > > > -- > Thanks, > Andrew Svetlov > ___ > Python-checkins mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-checkins -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to
On 19 Mar 2013, at 17:26, Antoine Pitrou wrote: > On Wed, 20 Mar 2013 01:22:58 +0100 (CET) > michael.foord wrote: >> http://hg.python.org/cpython/rev/684b75600fa9 >> changeset: 82811:684b75600fa9 >> user:Michael Foord >> date:Tue Mar 19 17:22:51 2013 -0700 >> summary: >> Closes issue 17467. Add readline and readlines support to >> unittest.mock.mock_open > > Wasn't it possible to re-use an existing implementation (such as > TextIOBase or StringIO) rather than re-write your own? > > (it's not even obvious your implementation is correct, BTW. How about > universal newlines?) mock_open makes it easy to put a StringIO in place if that's what you want. It's just a simple helper function for providing some known data *along with the Mock api* to make asserts that it was used correctly. It isn't presenting a full file-system. My suggestion to the implementor of the patch was that read / readline / readlines be disconnected - but the patch provided allows them to be interleaved and I saw no reason to undo that. If users want more complex behaviour (like universal newline support) they can use mock_open along with a StringIO. Michael > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to
On Tue, Mar 19, 2013 at 9:44 PM, Michael Foord wrote: > > On 19 Mar 2013, at 17:26, Antoine Pitrou wrote: > > > On Wed, 20 Mar 2013 01:22:58 +0100 (CET) > > michael.foord wrote: > >> http://hg.python.org/cpython/rev/684b75600fa9 > >> changeset: 82811:684b75600fa9 > >> user:Michael Foord > >> date:Tue Mar 19 17:22:51 2013 -0700 > >> summary: > >> Closes issue 17467. Add readline and readlines support to > unittest.mock.mock_open > > > > Wasn't it possible to re-use an existing implementation (such as > > TextIOBase or StringIO) rather than re-write your own? > > > > (it's not even obvious your implementation is correct, BTW. How about > > universal newlines?) > > mock_open makes it easy to put a StringIO in place if that's what you > want. It's just a simple helper function for providing some known data > *along with the Mock api* to make asserts that it was used correctly. It > isn't presenting a full file-system. My suggestion to the implementor of > the patch was that read / readline / readlines be disconnected - but the > patch provided allows them to be interleaved and I saw no reason to undo > that. > > If users want more complex behaviour (like universal newline support) they > can use mock_open along with a StringIO. > It'd be good to mention that in the unittest.mock.rst docs. > > Michael > > > > > > Regards > > > > Antoine. > > > > > > ___ > > Python-Dev mailing list > > [email protected] > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > > -- > http://www.voidspace.org.uk/ > > > May you do good and not evil > May you find forgiveness for yourself and forgive others > May you share freely, never taking more than you give. > -- the sqlite blessing > http://www.sqlite.org/different.html > > > > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
