Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Paul Sokolovsky 
Hello,

On Tue, 10 Jun 2014 13:03:03 +1000
Steven D'Aprano  wrote:

> On Tue, Jun 10, 2014 at 05:23:12AM +0300, Paul Sokolovsky wrote:
> 
> > execfile() builtin function was removed in 3.0. This brings few
> > problems:
> > 
> > 1. It hampers interactive mode - instead of short and easy to type
> > execfile("file.py") one needs to use exec(open("file.py").read()).
> 
> If the amount of typing is the problem, that's easy to solve:
> 
> # do this once
> def execfile(name):
> exec(open("file.py").read())

So, you here propose to workaround removal of core language feature
either a) on end user side, or b) on "system integrator" side. But such
solution is based on big number of assumptions, like: user wants to
workaround that at all (hint: they don't, they just want to use it);
you say "do this once", but actually it's "do it in each interactive
session again and again", and user may not have knowledge to "do it
once" instead; that if system integrator does that, the the function is
called "execfile": if system integrator didn't have enough Python
experience, and read only Python3 spec, they might call it something
else, and yet users with a bit of Python experience will expect it be
called exactly "execfile" and not anything else.

> 
> Another possibility is:
> 
> os.system("python file.py")
> 
> 
> > 2. Ok, assuming that exec(open().read()) idiom is still a way to go,
> > there's a problem - it requires to load entire file to memory. But
> > there can be not enough memory. Consider 1Mb file with 900Kb
> > comments (autogenerated, for example). execfile() could easily
> > parse it, using small buffer. But exec() requires to slurp entire
> > file into memory, and 1Mb is much more than heap sizes that we
> > target.
> 
> There's nothing stopping alternative implementations having their own 
> implementation-specific standard library modules.

And here you propose to workaround it on particular implementation's
level. But in my original mail, in excerpt that you removed, I kindly
asked to skip obvious suggestions (like that particular implementation
can do anything it wants).

I don't see how working around the issue on user, particular
distribution, or particular implementation level help *Python* language
in general, and *Python community* in general. So, any bright ideas how
to workaround the issue of execfile() removal on *language level*?

[]
> So you could do this:
> 
> from upy import execfile
> execfile("file.py")
> 
> So long as you make it clear that this is a platform specific module, 
> and don't advertise it as a language feature, I see no reason why you 
> cannot do that.

The case we discuss is clearly different. It's not about "platform
specific module", it's about functionality which was in Python all the
time, and was suddenly removed in Python3, for not fully clear, or
alternatively, not severe enough, reasons. If some implementation is to
re-add it, the description like above seems the most truthful way to
represent that function.


-- 
Best regards,
 Paul  mailto:[email protected]
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character

2014-06-14 Thread anatoly techtonik 
On Fri, Jun 13, 2014 at 2:55 AM, Ryan Gonzalez  wrote:

> SHELLS ARE NOT CROSS-PLATFORM Seriously, there are going to be
> differences. If you really must:
>
> escape = lambda s: s.replace('^', '^^') if os.name == 'nt' else s
>

It is not about generic shell problem, it is about specific behavior that
on Windows Python already uses cmd.exe shell hardcoded in its sources. So
for crossplatform behavior on Windows, it should escape symbols on command
passed to cmd.exe that are special to this shell to avoid breaking Python
scripts. What you propose is a bad workaround, because it assumes that all
Python users who use subprocess to execute hg or git should possess apriori
knowledge about default subprocess behaviour with default shell on Windows
and implement workaround for that.
-- 
anatoly t.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character

2014-06-14 Thread anatoly techtonik 
On Fri, Jun 13, 2014 at 5:11 AM, Nikolaus Rath  wrote:

> "R. David Murray"  writes:
> > Also notice that using a list with shell=True is using the API
> > incorrectly.  It wouldn't even work on Linux, so that torpedoes
> > the cross-platform concern already :)
> >
> > This kind of confusion is why I opened http://bugs.python.org/issue7839.
>
> Can someone describe an use case where shell=True actually makes sense
> at all?
>

You need to write a wrapper script to automate several user commands. It is
quite common to use shell pipe redirection for joining many utils and calls
together than to rewrite data pipes in Python.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character

2014-06-14 Thread anatoly techtonik 
On Thu, Jun 12, 2014 at 5:12 AM, Chris Angelico  wrote:

> On Thu, Jun 12, 2014 at 12:07 PM, Chris Angelico  wrote:
> > ISTM what you want is not shell=True, but a separate function that
> > follows the system policy for translating a command name into a
> > path-to-binary. That's something that, AFAIK, doesn't currently exist
> > in the Python 2 stdlib, but Python 3 has shutil.which(). If there's a
> > PyPI backport of that for Py2, you should be able to use that to
> > figure out the command name, and then avoid shell=False.
>
> Huh. Next time, Chris, search the web before you post. Via a
> StackOverflow post, learned about distutils.spawn.find_executable().
>

I remember I even wrote a patch for it, but I forgot about it already.
Still feels like a hack that is difficult to find and understand that you
need really it. In Rietveld case it won't work, because upload.py script
allows user to specify arbitrary diff command to send change for
review.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Paul Sokolovsky 
Hello,

On Tue, 10 Jun 2014 17:36:02 +1000
Nick Coghlan  wrote:

> On 10 June 2014 12:23, Paul Sokolovsky  wrote:
> > 1. It hampers interactive mode - instead of short and easy to type
> > execfile("file.py") one needs to use exec(open("file.py").read()).
> > I'm sure that's not going to bother a lot of people - after all, the
> > easiest way to execute a Python file is to drop back to shell and
> > restart python with file name, using all wonders of tab completion.
> > But now imagine that Python interpreter runs on bare hardware, and
> > its REPL is the only shell. That's exactly what we have with
> > MicroPython's Cortex-M port. But it's not really
> > MicroPython-specific, there's CPython port to baremetal either -
> > http://www.pycorn.org/ .
> 
> https://docs.python.org/3/library/runpy.html#runpy.run_path
> 
> import runpy
> file_globals = runpy.run_path("file.py")

Thanks, it's the most productive response surely. So, at least there's
alternative to removed execfile(). Unfortunately, I don't think it's
good alternative to execfile() in all respects. It clearly provides API
for that functionality, but is that solution of least surprise and is
it actually known by users at all (to be useful for them)?

Googling for "execfile python 3", top 3 hits I see are stackoverflow
questions, *none* of which mentions runpy. So, people either don't
consider it viable alternative to execfile, or don't know about it at
all (my guess it's the latter).


Like with previous discussion, its meaning goes beyond just Python
realm - there's competition all around. And internets bring funny
examples, like for example http://www.red-lang.org/p/contributions.html
(scroll down to diagram, or here's direct link:
http://3.bp.blogspot.com/-xhOP35Dm99w/UuXFKgY2dlI/AGA/YQu98_pPDjw/s1600/reichart-abstraction-diagram.png)
So, didn't you know that Ruby can be used for OS-level development, and
Python can't? Or that JavaScript DSL capabilities are better than
Python's (that's taking into account that JavaScript DSL capabilities
are represented by JSON, whose creators were so arrogant as to disallow
even usage of comments in it).

So, now suppose there's a discussion of how good different languages are
for interactive usage (out of the box apparently). It would be a little
hard to defend claim that Python is *excellent* interactive language,
if its latest series got -1 on that scale, by removing feature which
may be indispensable at times. Knowing that, one subconsciously may
start to wonder if Ruby or JavaScript are doing it (in wide sense)
better than Python.


--
Best regards,
 Paul  mailto:[email protected]
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Markus Unterwaditzer 
On Tue, Jun 10, 2014 at 05:23:12AM +0300, Paul Sokolovsky wrote:
> Hello,
> 
> I was pleasantly surprised with the response to recent post about
> MicroPython implementation details 
> (https://mail.python.org/pipermail/python-dev/2014-June/134718.html). I
> hope that discussion means that posts about alternative implementations
> are not unwelcome here, so I would like to bring up another (of many)
> issues we faced while implementing MicroPython.
> 
> execfile() builtin function was removed in 3.0. This brings few
> problems:
> 
> 1. It hampers interactive mode - instead of short and easy to type
> execfile("file.py") one needs to use exec(open("file.py").read()). I'm
> sure that's not going to bother a lot of people - after all, the
> easiest way to execute a Python file is to drop back to shell and
> restart python with file name, using all wonders of tab completion. But
> now imagine that Python interpreter runs on bare hardware, and its REPL
> is the only shell. That's exactly what we have with MicroPython's
> Cortex-M port. But it's not really MicroPython-specific, there's
> CPython port to baremetal either - http://www.pycorn.org/ .

As far as i can see, minimizing the amount of characters to type was never a
design goal of the Python language. And because that goal never mattered as
much for the designers as it seems to do for you, the reason for it to get
removed -- reducing the amount of builtins without reducing functionality --
was the only one left.

> 2. Ok, assuming that exec(open().read()) idiom is still a way to go,
> there's a problem - it requires to load entire file to memory. But
> there can be not enough memory. Consider 1Mb file with 900Kb comments
> (autogenerated, for example). execfile() could easily parse it, using
> small buffer. But exec() requires to slurp entire file into memory, and
> 1Mb is much more than heap sizes that we target.

That is a valid concern, but i believe violating the language specification and
adding your own execfile implementation (either as a builtin or in a new stdlib
module) here is justified, even if it means you will have to modify your
existing Python 3 code to use it -- i don't think the majority of software
written in Python will be able to run under such memory constraints without
major modifications anyway.

> Comments, suggestions? Just to set a productive direction, please
> kindly don't consider the problems above as MicroPython's.

A new (not MicroPython-specific) stdlib module containing functions such as
execfile could be considered. Not really for Python-2-compatibility, but for
performance-critical situations.

I am not sure if this is a good solution. Not at all. Even though it's
separated from the builtins, i think it would still sacrifice the purity of the
the language (by which i mean having a minimal composable API), because people
are going to use it anyway. It reminds me of the situation in Python 2 where
developers are trying to use cStringIO with a fallback to StringIO as a matter
of principle, not because they actually need that kind of performance.

Another, IMO better idea which shifts the problem to the MicroPython devs is to
"just" detect code using

exec(open(...).read())

and transparently rewrite it to something more memory-efficient. This is the
idea i actually think is a good one.


> I very much liked how last discussion went: I was pointed that
> https://docs.python.org/3/reference/index.html is not really a CPython
> reference, it's a *Python* reference, and there were even motion to
> clarify in it some points which came out from MicroPython discussion.
> So, what about https://docs.python.org/3/library/index.html - is it
> CPython, or Python standard library specification? Assuming the latter,
> what we have is that, by removal of previously available feature,
> *Python* became less friendly for interactive usage and less scalable.

"Less friendly for interactive usage" is a strong and vague statement. If
you're going after the amount of characters required to type, yes, absolutely,
but by that terms one could declare Bash and Perl to be superior languages.
Look at it from a different perspective: There are fewer builtins to remember.

> 
> 
> Thanks,
>  Paul  mailto:[email protected]
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/markus%40unterwaditzer.net
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Fabio Zadrozny 
On Sat, Jun 14, 2014 at 6:00 PM, Markus Unterwaditzer <
[email protected]> wrote:

> On Tue, Jun 10, 2014 at 05:23:12AM +0300, Paul Sokolovsky wrote:
> > Hello,
> >
> > I was pleasantly surprised with the response to recent post about
> > MicroPython implementation details
> > (https://mail.python.org/pipermail/python-dev/2014-June/134718.html). I
> > hope that discussion means that posts about alternative implementations
> > are not unwelcome here, so I would like to bring up another (of many)
> > issues we faced while implementing MicroPython.
> >
> > execfile() builtin function was removed in 3.0. This brings few
> > problems:
> >
> > 1. It hampers interactive mode - instead of short and easy to type
> > execfile("file.py") one needs to use exec(open("file.py").read()). I'm
> > sure that's not going to bother a lot of people - after all, the
> > easiest way to execute a Python file is to drop back to shell and
> > restart python with file name, using all wonders of tab completion. But
> > now imagine that Python interpreter runs on bare hardware, and its REPL
> > is the only shell. That's exactly what we have with MicroPython's
> > Cortex-M port. But it's not really MicroPython-specific, there's
> > CPython port to baremetal either - http://www.pycorn.org/ .
>
> As far as i can see, minimizing the amount of characters to type was never
> a
> design goal of the Python language. And because that goal never mattered as
> much for the designers as it seems to do for you, the reason for it to get
> removed -- reducing the amount of builtins without reducing functionality
> --
> was the only one left.
>
> > 2. Ok, assuming that exec(open().read()) idiom is still a way to go,
> > there's a problem - it requires to load entire file to memory. But
> > there can be not enough memory. Consider 1Mb file with 900Kb comments
> > (autogenerated, for example). execfile() could easily parse it, using
> > small buffer. But exec() requires to slurp entire file into memory, and
> > 1Mb is much more than heap sizes that we target.
>
> That is a valid concern, but i believe violating the language
> specification and
> adding your own execfile implementation (either as a builtin or in a new
> stdlib
> module) here is justified, even if it means you will have to modify your
> existing Python 3 code to use it -- i don't think the majority of software
> written in Python will be able to run under such memory constraints without
> major modifications anyway.
>
> > Comments, suggestions? Just to set a productive direction, please
> > kindly don't consider the problems above as MicroPython's.
>
> A new (not MicroPython-specific) stdlib module containing functions such as
> execfile could be considered. Not really for Python-2-compatibility, but
> for
> performance-critical situations.
>
> I am not sure if this is a good solution. Not at all. Even though it's
> separated from the builtins, i think it would still sacrifice the purity
> of the
> the language (by which i mean having a minimal composable API), because
> people
> are going to use it anyway. It reminds me of the situation in Python 2
> where
> developers are trying to use cStringIO with a fallback to StringIO as a
> matter
> of principle, not because they actually need that kind of performance.
>
> Another, IMO better idea which shifts the problem to the MicroPython devs
> is to
> "just" detect code using
>
> exec(open(...).read())
>
> and transparently rewrite it to something more memory-efficient. This is
> the
> idea i actually think is a good one.
>
>
> > I very much liked how last discussion went: I was pointed that
> > https://docs.python.org/3/reference/index.html is not really a CPython
> > reference, it's a *Python* reference, and there were even motion to
> > clarify in it some points which came out from MicroPython discussion.
> > So, what about https://docs.python.org/3/library/index.html - is it
> > CPython, or Python standard library specification? Assuming the latter,
> > what we have is that, by removal of previously available feature,
> > *Python* became less friendly for interactive usage and less scalable.
>
> "Less friendly for interactive usage" is a strong and vague statement. If
> you're going after the amount of characters required to type, yes,
> absolutely,
> but by that terms one could declare Bash and Perl to be superior languages.
> Look at it from a different perspective: There are fewer builtins to
> remember.
>
> >
>

Well, I must say that the exec(open().read()) is not really a proper
execfile implementation because it may fail because of encoding issues...
(i.e.: one has to check the file encoding to do the open with the proper
encoding, otherwise it's possible to end up with gibberish).

The PyDev debugger has an implementation (see:
https://github.com/fabioz/Pydev/blob/development/plugins/org.python.pydev/pysrc/_pydev_execfile.py)
which considers the encoding so that the result is ok (but it still has a
bug related to utf-8 w

[Python-Dev] Why does _pyio.*.readinto have to work with 'b' arrays?

2014-06-14 Thread Nikolaus Rath 
Hello,

The _pyio.BufferedIOBase class contains the following hack to make sure
that you can read-into array objects with format 'b':

try:
b[:n] = data
except TypeError as err:
import array
if not isinstance(b, array.array):
raise err
b[:n] = array.array('b', data)

I am now wondering if I should implement the same hack in BufferedReader
(cf. issue 20578). Is there anything special about 'b' arrays that
justifies to treat them this way?

Note that readinto is supposed to work with any object implementing the
buffer protocol, but the Python implementation only works with
bytearrays and (with the above hack) 'b' arrays. Even using a 'B' array
fails:

>>> import _pyio
>>> from array import array
>>> buf = array('b', b'x' * 10)
>>> _pyio.open('/dev/zero', 'rb').readinto(buf) 
10
>>> buf = array('B', b'x' * 10)
>>> _pyio.open('/dev/zero', 'rb').readinto(buf)
Traceback (most recent call last):
  File "/home/nikratio/clones/cpython/Lib/_pyio.py", line 662, in readinto
b[:n] = data
TypeError: can only assign array (not "bytes") to array slice

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 1, in 
  File "/home/nikratio/clones/cpython/Lib/_pyio.py", line 667, in readinto
b[:n] = array.array('b', data)
TypeError: bad argument type for built-in operation


It seems to me that a much cleaner solution would be to simply declare
_pyio's readinto to only work with bytearrays, and to explicitly raise a
(more helpful) TypeError if anything else is passed in.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Greg Ewing 

Fabio Zadrozny wrote:
Well, I must say that the exec(open().read()) is not really a proper 
execfile implementation because it may fail because of encoding 
issues...


It's not far off, though -- all it needs is an optional
encoding parameter.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Steve Dower 
I think the point is that the encoding may be embedded in the file as a coding 
comment and there's no obvious way to deal with that.

Top-posted from my Windows Phone

From: Greg Ewing
Sent: ‎6/‎14/‎2014 16:19
To: [email protected]
Subject: Re: [Python-Dev] Criticism of execfile() removal in Python3

Fabio Zadrozny wrote:
> Well, I must say that the exec(open().read()) is not really a proper
> execfile implementation because it may fail because of encoding
> issues...

It's not far off, though -- all it needs is an optional
encoding parameter.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Skip Montanaro 
> you say "do this once", but actually it's "do it in each interactive
> session again and again",  ...

That's what your Python startup file is for. I have been running with
several tweaked builtin functions for years. Never have to consciously load
them. If I wanted execfile badly enough, I'd define it there.

I don't think I've used execfile more than a handful of times in the 20-odd
years I've been using Python. Perhaps our personal approaches to executing
code at the interpreter prompt are radically different, but I think if the
lack of execfile is such a big deal for you, you might want to check around
to see how other people use interactive mode.

Skip
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does _pyio.*.readinto have to work with 'b' arrays?

2014-06-14 Thread Benjamin Peterson 
On Sat, Jun 14, 2014, at 15:39, Nikolaus Rath wrote:
> It seems to me that a much cleaner solution would be to simply declare
> _pyio's readinto to only work with bytearrays, and to explicitly raise a
> (more helpful) TypeError if anything else is passed in.

That seems reasonable. I don't think _pyio's behavior is terribly
important compared to the C _io module.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Nick Coghlan 
On 15 Jun 2014 06:52, "Paul Sokolovsky"  wrote:
>
> Hello,
>
> On Tue, 10 Jun 2014 17:36:02 +1000
> Nick Coghlan  wrote:
>
> > On 10 June 2014 12:23, Paul Sokolovsky  wrote:
> > > 1. It hampers interactive mode - instead of short and easy to type
> > > execfile("file.py") one needs to use exec(open("file.py").read()).
> > > I'm sure that's not going to bother a lot of people - after all, the
> > > easiest way to execute a Python file is to drop back to shell and
> > > restart python with file name, using all wonders of tab completion.
> > > But now imagine that Python interpreter runs on bare hardware, and
> > > its REPL is the only shell. That's exactly what we have with
> > > MicroPython's Cortex-M port. But it's not really
> > > MicroPython-specific, there's CPython port to baremetal either -
> > > http://www.pycorn.org/ .
> >
> > https://docs.python.org/3/library/runpy.html#runpy.run_path
> >
> > import runpy
> > file_globals = runpy.run_path("file.py")
>
> Thanks, it's the most productive response surely. So, at least there's
> alternative to removed execfile(). Unfortunately, I don't think it's
> good alternative to execfile() in all respects. It clearly provides API
> for that functionality, but is that solution of least surprise and is
> it actually known by users at all (to be useful for them)?

We don't want people instinctively reaching for execfile (or run_path for
that matter). It's almost always the wrong answer to a problem (because it
runs code in a weird, ill-defined environment and has undefined behaviour
when used inside a function), meeting the definition of "attractive
nuisance".

We moved reload() to imp.reload() and reduce() to functools.reduce() for
similar reasons - they're too rarely the right answer to justify having
them globally available by default.

> Googling for "execfile python 3", top 3 hits I see are stackoverflow
> questions, *none* of which mentions runpy. So, people either don't
> consider it viable alternative to execfile, or don't know about it at
> all (my guess it's the latter).

Given the relative age of the two APIs, that seems likely. Adding answers
pointing users to the runpy APIs could be useful.

> Like with previous discussion, its meaning goes beyond just Python
> realm - there's competition all around. And internets bring funny
> examples, like for example http://www.red-lang.org/p/contributions.html
> (scroll down to diagram, or here's direct link:
>
http://3.bp.blogspot.com/-xhOP35Dm99w/UuXFKgY2dlI/AGA/YQu98_pPDjw/s1600/reichart-abstraction-diagram.png
)
> So, didn't you know that Ruby can be used for OS-level development, and
> Python can't? Or that JavaScript DSL capabilities are better than
> Python's (that's taking into account that JavaScript DSL capabilities
> are represented by JSON, whose creators were so arrogant as to disallow
> even usage of comments in it).

There's a lot of misinformation on the internet. While there is certainly
room for the PSF to do more in terms of effectively communicating Python's
ubiquity and strengths (and we're working on that), "people with no clue
post stuff on the internet" doesn't make a compelling *technical* argument
(which is what is needed to get new builtins added).

> So, now suppose there's a discussion of how good different languages are
> for interactive usage (out of the box apparently). It would be a little
> hard to defend claim that Python is *excellent* interactive language,
> if its latest series got -1 on that scale, by removing feature which
> may be indispensable at times. Knowing that, one subconsciously may
> start to wonder if Ruby or JavaScript are doing it (in wide sense)
> better than Python.

Yes, people get upset when we tell them we consider some aspects of their
software designs to be ill-advised. Running other code in the *current*
namespace is such a thing - it is typically preferable to run it in a
*different* namespace and then access the results, rather than implicitly
overwriting the contents of the current namespace.

That said, a question still worth asking is whether there is scope for
additional runpy APIs that are designed to more easily implement Python 2
and IPython style modes of operation where independent units of code
manipulate a shared namespace? That's actually a possibility, but any such
proposals need to be presented on python-ideas in terms of the *use case*
to be addressed, rather than the fact that execfile() happened to be the
preferred solution in Python 2.

Regards,
Nick.

>
>
> --
> Best regards,
>  Paul  mailto:[email protected]
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Nick Coghlan 
On 15 Jun 2014 09:37, "Steve Dower"  wrote:
>
> I think the point is that the encoding may be embedded in the file as a
coding comment and there's no obvious way to deal with that.

Opening source files correctly is the intended use case for tokenize.open().

Cheers,
Nick.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character

2014-06-14 Thread Ryan Gonzalez 
Of course cmd.exe is hardcoded; there are no other shells on Windows! (I'm
purposely ignoring MinGW, Cygwin, command.com, etc.) If anything,
auto-escaping will break scripts that are already designed to escape carets
on Windows.


On Sat, Jun 14, 2014 at 2:54 PM, anatoly techtonik 
wrote:

> On Fri, Jun 13, 2014 at 2:55 AM, Ryan Gonzalez  wrote:
>
>> SHELLS ARE NOT CROSS-PLATFORM Seriously, there are going to be
>> differences. If you really must:
>>
>> escape = lambda s: s.replace('^', '^^') if os.name == 'nt' else s
>>
>
> It is not about generic shell problem, it is about specific behavior that
> on Windows Python already uses cmd.exe shell hardcoded in its sources. So
> for crossplatform behavior on Windows, it should escape symbols on command
> passed to cmd.exe that are special to this shell to avoid breaking Python
> scripts. What you propose is a bad workaround, because it assumes that all
> Python users who use subprocess to execute hg or git should possess apriori
> knowledge about default subprocess behaviour with default shell on Windows
> and implement workaround for that.
>  --
> anatoly t.
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character

2014-06-14 Thread Greg Ewing 

On Thu, Jun 12, 2014 at 12:07 PM, Chris Angelico mailto:[email protected]>> wrote:
 > ISTM what you want is not shell=True, but a separate function that
 > follows the system policy for translating a command name into a
 > path-to-binary.


According to the docs, subprocess.Popen should already be
doing this on Unix:

   On Unix, with shell=False (default): In this case, the Popen class
   uses os.execvp() to execute the child program.

and execvp() searches the user's PATH to find the program.

However, it says the Windows version uses CreateProcess, which
doesn't use PATH.

This seems like an unfortunate platform difference to me. It
would be better if PATH were searched on both platforms, or
better still, make it an option independent of shell=True.

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Steve Dower 
So is exec(tokenize.open(file).read()) the actual replacement for execfile()? 
Not too bad, but still not obvious (or widely promoted - I'd never heard of it).

Top-posted from my Windows Phone

From: Nick Coghlan
Sent: ‎6/‎14/‎2014 18:31
To: Steve Dower
Cc: Greg Ewing; 
[email protected]
Subject: Re: [Python-Dev] Criticism of execfile() removal in Python3


On 15 Jun 2014 09:37, "Steve Dower" 
mailto:[email protected]>> wrote:
>
> I think the point is that the encoding may be embedded in the file as a 
> coding comment and there's no obvious way to deal with that.

Opening source files correctly is the intended use case for tokenize.open().

Cheers,
Nick.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does _pyio.*.readinto have to work with 'b' arrays?

2014-06-14 Thread Nick Coghlan 
On 15 June 2014 10:41, Benjamin Peterson  wrote:
> On Sat, Jun 14, 2014, at 15:39, Nikolaus Rath wrote:
>> It seems to me that a much cleaner solution would be to simply declare
>> _pyio's readinto to only work with bytearrays, and to explicitly raise a
>> (more helpful) TypeError if anything else is passed in.
>
> That seems reasonable. I don't think _pyio's behavior is terribly
> important compared to the C _io module.

_pyio was written before the various memoryview fixes that were
implemented in Python 3.3 - it seems to me it would make more sense to
use memoryview to correctly handle arbitrary buffer exporters (we
implemented similar fixes for the base64 module in 3.4).

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Criticism of execfile() removal in Python3

2014-06-14 Thread Nick Coghlan 
On 15 June 2014 13:15, Steve Dower  wrote:
> So is exec(tokenize.open(file).read()) the actual replacement for
> execfile()? Not too bad, but still not obvious (or widely promoted - I'd
> never heard of it).

Yes, that's pretty close. It's still a dubious idea due to the
implicit modification of the local namespace (and the resulting
differences in behaviour at function level due to the fact that
writing to locals() doesn't actually update the local namespace).

That said, the "implicit changes to the local namespace are a bad
idea" concern applies to exec() in general, so it was the "it's just a
shorthand for a particular use of exec" aspect that tipped in the
balance in the demise of execfile (this is also implied by the
phrasing of the relevant bullet point in PEP 3100).

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does _pyio.*.readinto have to work with 'b' arrays?

2014-06-14 Thread Nikolaus Rath 
On 06/14/2014 09:31 PM, Nick Coghlan wrote:
> On 15 June 2014 10:41, Benjamin Peterson  wrote:
>> On Sat, Jun 14, 2014, at 15:39, Nikolaus Rath wrote:
>>> It seems to me that a much cleaner solution would be to simply declare
>>> _pyio's readinto to only work with bytearrays, and to explicitly raise a
>>> (more helpful) TypeError if anything else is passed in.
>>
>> That seems reasonable. I don't think _pyio's behavior is terribly
>> important compared to the C _io module.
> 
> _pyio was written before the various memoryview fixes that were
> implemented in Python 3.3 - it seems to me it would make more sense to
> use memoryview to correctly handle arbitrary buffer exporters (we
> implemented similar fixes for the base64 module in 3.4).

Definitely. But is there a way to do that without writing C code?

My attempts failed:

>>> from array import array
>>> a = array('b', b'x'*10)
>>> am = memoryview(a)
>>> am[:3] = b'foo'
Traceback (most recent call last):
  File "", line 1, in 
ValueError: memoryview assignment: lvalue and rvalue have different
structures
>>> am[:3] = memoryview(b'foo')
Traceback (most recent call last):
  File "", line 1, in 
ValueError: memoryview assignment: lvalue and rvalue have different
structures
>>> am.format = 'B'
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: attribute 'format' of 'memoryview' objects is not writable

The only thing that works is:

>>> am[:3] = array('b', b'foo')

but that's again specific to a being a 'b'-array.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character

2014-06-14 Thread Tim Golden 

On 15/06/2014 02:22, Ryan Gonzalez wrote:

Of course cmd.exe is hardcoded;


Of course it's not:

(from Lib/subprocess.py)

comspec = os.environ.get("COMSPEC", "cmd.exe")

I don't often expect, in these post-command.com days, to get anything 
other than cmd.exe. But alternative command processors are certainly 
possible.


TJG
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why does _pyio.*.readinto have to work with 'b' arrays?

2014-06-14 Thread Nick Coghlan 
On 15 June 2014 14:57, Nikolaus Rath  wrote:
> On 06/14/2014 09:31 PM, Nick Coghlan wrote:
>> On 15 June 2014 10:41, Benjamin Peterson  wrote:
>>> On Sat, Jun 14, 2014, at 15:39, Nikolaus Rath wrote:
 It seems to me that a much cleaner solution would be to simply declare
 _pyio's readinto to only work with bytearrays, and to explicitly raise a
 (more helpful) TypeError if anything else is passed in.
>>>
>>> That seems reasonable. I don't think _pyio's behavior is terribly
>>> important compared to the C _io module.
>>
>> _pyio was written before the various memoryview fixes that were
>> implemented in Python 3.3 - it seems to me it would make more sense to
>> use memoryview to correctly handle arbitrary buffer exporters (we
>> implemented similar fixes for the base64 module in 3.4).
>
> Definitely. But is there a way to do that without writing C code?

Yes, Python level reshaping and typecasting of memory views is one of
the key enhancements Stefan implemented for 3.3.

>>> from array import array
>>> a = array('b', b'x'*10)
>>> am = memoryview(a)
>>> a
array('b', [120, 120, 120, 120, 120, 120, 120, 120, 120, 120])
>>> am[:3] = memoryview(b'foo').cast('b')
>>> a
array('b', [102, 111, 111, 120, 120, 120, 120, 120, 120, 120])

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com