Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
Nick Coghlan writes: > str and bytes aren't going to implement __fspath__ (since they're > only *sometimes* path objects), so asking people to call the > protocol method directly for any purpose would be a pain. It *should* be a pain. People who need bytes should call fsencode, people who need str should call fsdecode, and Ethan's antipathy checks for bytes and str, then calls __fspath__ if needed. Who's left? Just the bartender and the janitor, last call was hours ago. OK, maybe there are enough clients to make it worthwhile to provide the utility, but it should be clearly marked as "double opt-in, for experts only (consenting adults must show proof of insurance)". The functionality of raising on wrong types can be incorporated in fsencode and fsdecode, but I think there's still some discussion needed about the conditions for raising, and what flags are needed. Of course with this reinterpretation, names like "fs_ensure_str" and "fs_ensure_bytes" might be more appropriate (much as y'all hate putting types in function names, in this case I think that's best). But backward compatibility, and the existing names aren't *that* bad I guess. > You may have missed my email where I agreed os.fspath() itself > needs to ensure the output is a str object and throw an exception > otherwise. Presumably it should do the same for bytes when those are desired, though. I don't find the "cast to bytes using memoryview" approach plausible, especially not where I live: if str, very likely some of the characters will be outside of the latin1 repertoire, and thus the internal representation will likely be full of NULs, and certainly not be what the user wants. > The remaining API design debate relates to whether the polymorphic > version should be "os.fspath(obj, allow_bytes=True)" or > "os._raw_fspath(obj)" (with Ethan favouring the former, and me the > latter). > > Et tu, Nick? "Guarantee"?! You can't guarantee any such thing > > with an implicitly invoked polymorphic API like this one -- > > unless you consider a crashed program to be in the binary > > domain. ;-) > > I do, as one of the core changes in design philosophy between > Python 2 and 3 is attempting to remove the implicit level shifting > between the binary and text domains, Hey, Reverend, I've been singing those hymns since the early '90s. > and instead throw exceptions in those cases. Then I don't understand the current design of fsdecode and fsencode. Shouldn't they raise on str and bytes respectively, rather than passing them through? In general, I would expect that something that's explicitly intended to be polymorphic would be documented as such, and the *caller* would be responsible for type-checking and raising if it got the wrong thing. Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
On 17 April 2016 at 18:03, Stephen J. Turnbull wrote: > Nick Coghlan writes: > > and instead throw exceptions in those cases. > > Then I don't understand the current design of fsdecode and fsencode. > Shouldn't they raise on str and bytes respectively, rather than > passing them through? In general, I would expect that something > that's explicitly intended to be polymorphic would be documented as > such, and the *caller* would be responsible for type-checking and > raising if it got the wrong thing. > I was initially surprised myself, but then realised it made sense for their intended use cases - if almost every usage looks like "obj if isinstance(obj, str) else os.fsdecode(obj)", then there ends up being a strong pragmatic case for pushing the pass-through down into the underlying function to reduce code duplication and rejecting str input in the cases where it isn't supported. By contrast, there are lots of places where "obj.decode()" gets called without a pass-through for objects that are already decoded. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
On Sun, Apr 17, 2016 at 11:03 AM, Stephen J. Turnbull
wrote:
> Nick Coghlan writes:
>
> > str and bytes aren't going to implement __fspath__ (since they're
> > only *sometimes* path objects), so asking people to call the
> > protocol method directly for any purpose would be a pain.
>
> It *should* be a pain. People who need bytes should call fsencode,
> people who need str should call fsdecode, and Ethan's antipathy checks
> for bytes and str, then calls __fspath__ if needed. Who's left? Just
> the bartender and the janitor, last call was hours ago. OK, maybe
> there are enough clients to make it worthwhile to provide the utility,
> but it should be clearly marked as "double opt-in, for experts only
> (consenting adults must show proof of insurance)".
My doubts, expressed several times in these threads, about the need
for a *public* os.fspath function to complement the __fspath__
protocol, are now perhaps gone. I'll explain why (and how). The
reasons for my doubts were that
(1) The audience outside the stdlib for such a function should be
small, because it is preferred to either use existing tools in
os.path.* or pathlib (or similar) for manipulating paths.
(2) There are just too many different possible versions of this
function: rejecting str, rejecting bytes, coercion to str, coercion to
bytes, and accepting both str and bytes. That's a total of 5 different
cases. People also used to talk about versions that would not allow
passing through objects that are already bytes or str. That would make
it a total of 10 different versions!
(in principle, there could be even more, but let's not go there :-).
In other words, this argument was that it is probably best to
implement whatever flavor is needed for the context, perhaps based on
documented recipes.
Regarding (2), we can first rule out half of the 10 cases---the ones
that reject plain instances of bytes and/or str---because they would
not be very useful as all the isinstance/hasattr checking etc. would
be left to the caller. And here are the remaining five, explained
based on what they accept as argument, what they return, and where
they would be used:
(A) "polymorphic"
*Accept*: str and bytes, provided via __fspath__ as well as plain str
and bytes instances.
*Return*: str/bytes depending on input.
*Audience*: the stdlib, including os.path.things, os.things,
shutil.things, open, ... (some functions would need a C version).
There may even be a small audience outside the stdlib.
(B) "str-based only"
*Accept*: str, provided via __fspath__ as well as plain str.
*Return*: str.
*Audience*: relatively low-level code that works exclusively with str
paths but accepts specialized path objects as input.
(C) "bytes-based only"
*Accept*: bytes, provided via __fspath__ as well as plain bytes.
*Return*: bytes.
*Audience*: low-level code that explicitly deals with paths as bytes
(probably to deal with undefined/ill-defined encodings).
(D) "coerce to str"
*Accept*: str and bytes, provided via __fspath__ as well as plain str
and bytes instances.
*Return*: str (coerced / decoded if needed).
*Audience*: code that deals explicitly with str but wants to 'try'
supporting bytes-based path inputs too via implicit decoding (even if
it may result in surrogate escapes, which one cannot for instance
print(...).)
(E) "coerce to bytes"
*Accept*: str and bytes, provided via __fspath__ as well as plain str
and bytes instances.
*Return*: bytes (coerced / encoded if needed).
*Audience*: low-level code that explicitly deals with bytes paths but
wants to accept str-based path inputs too via implicit encoding.
Even if all options (A-E) probably have small audiences (compared to
e.g. os.path.*), some of them have larger audiences than others. But
all of them have at least *some* reasonable audience (as desribed
above).
Recently (well, a few days ago, but 'recently', considering the scale
of these discussions anyway ;-), Nick pointed out something I hadn't
realized---os.fsencode and os.fsdecode actually already implement
coercion to bytes and str, respectively. With those two functions made
compatible with the __fspath__ protocol [using (A) above], they would
in fact *be* (D) and (E), respectively.
Now, we only have options (A-C) left. They could all be implemented
roughly as follows:
def fspath(pathlike, *, output_types = (str,)):
if hasattr(pathlike, '__fspath__'):
ret = pathlike.__fspath__() # or pathlike.__fspath__ if it's not a method
else:
ret = pathlike
if not isinstance(ret, output_types):
raise TypeError("argument is not and does not provide an
acceptable pathname")
return ret
With an implementation like the above, (A) would correspond to
output_types = (str, bytes), (B) to the default, and (C) to
output_types = (bytes,).
So, with the above considerations as a counterargument, I consider
argument (2) gone.
What about argument (1), that the audience for the os.fspath(...)
function (especially for one selected version of the 5 or 10
variation
Re: [Python-Dev] Wordcode: new regular bytecode using 16-bit units
Just on the off chance that it’s related, could it have something to do with the bug in findlabels? http://bugs.python.org/issue26448 (I have high confidence that my patch fixes the problem, just haven’t gotten around to completing the tests.) From: Demur Rumed [mailto:[email protected]] Sent: Saturday, April 16, 2016 17:05 To: [email protected] Subject: Re: [Python-Dev] Wordcode: new regular bytecode using 16-bit units The outstanding bug with this patch right now is a regression in line numbers causing the test for http://bugs.python.org/issue9936 to fail. I've tried to debug it without success ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
On 04/17/2016 06:58 AM, Koos Zevenhoven wrote: So, as a summary: With a str+bytes-polymorphic __fspath__, with the above argumentation and the rough implementation of os.fspath(...), the conclusion is that the os.fspath function should indeed be public, and that no further variations are needed. Nice summation, thank you. :) -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
On Sun, Apr 17, 2016 at 9:14 PM, Ethan Furman wrote: > On 04/17/2016 06:58 AM, Koos Zevenhoven wrote: > >> So, as a summary: With a str+bytes-polymorphic __fspath__, with the >> above argumentation and the rough implementation of os.fspath(...), >> the conclusion is that the os.fspath function should indeed be public, >> and that no further variations are needed. > > > Nice summation, thank you. :) > Come on, Ethan, that summary was not for you ;) It was for lazy people, people with bad memory, or people not so involved in the topic. I wrote a big post, provided new arguments, with other points collected into the same logical framework, wrote a new version of os.fspath and argued why it is the right one --- and all you do is read the stupid summary. You can do better than that: read the whole thing! ;-). -Koos ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
On Mon, Apr 18, 2016 at 7:05 AM, Koos Zevenhoven wrote: > On Sun, Apr 17, 2016 at 9:14 PM, Ethan Furman wrote: >> On 04/17/2016 06:58 AM, Koos Zevenhoven wrote: >> >>> So, as a summary: With a str+bytes-polymorphic __fspath__, with the >>> above argumentation and the rough implementation of os.fspath(...), >>> the conclusion is that the os.fspath function should indeed be public, >>> and that no further variations are needed. >> >> >> Nice summation, thank you. :) >> > > Come on, Ethan, that summary was not for you ;) It was for lazy > people, people with bad memory, or people not so involved in the > topic. I wrote a big post, provided new arguments, with other points > collected into the same logical framework, wrote a new version of > os.fspath and argued why it is the right one --- and all you do is > read the stupid summary. You can do better than that: read the whole > thing! ;-). Yes, but people like me who haven't read every single post appreciate the vote of support from someone who has. Ethan's post says that this one-paragraph summary has twice as much weight as it had when only one person attests it. So, thank you Koos for summarizing, and thank you Ethan for affirming the summary. ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pathlib enhancements - acceptable inputs and outputs for __fspath__ and os.fspath()
On 04/17/2016 02:05 PM, Koos Zevenhoven wrote: On Sun, Apr 17, 2016 at 9:14 PM, Ethan Furman wrote: On 04/17/2016 06:58 AM, Koos Zevenhoven wrote: So, as a summary: With a str+bytes-polymorphic __fspath__, with the above argumentation and the rough implementation of os.fspath(...), the conclusion is that the os.fspath function should indeed be public, and that no further variations are needed. Nice summation, thank you. :) Come on, Ethan, that summary was not for you ;) Heh. You can do better than that: read the whole thing! ;-). Ah, but I did read the whole thing! I just didn't want to quote it all and then add one line, so I snipped the rest. Let me try again: Good, well thought-out post. Thank you. :) if-at-first-you-don't-succeed'ly yrs, -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] My first post here ~ do you need more Python core developers on Windows?
Hi, I just subscribed to the "Python-Dev" mailing list and the 'Welcome" reply asked me to introduce myself. My name is Burkhard Meier and I wrote the "Python GUI Programming Cookbook" published by Packt. It is available on Amazon and PacktPub.com. Maybe I can become more involved in the Python community as a Python developer on Windows . Kind regards, Burkhard Meier ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
