Re: [Python-Dev] PEP 492: No new syntax is required

2015-04-27 Thread Mark Shannon 



On 27/04/15 00:13, Guido van Rossum wrote:

But new syntax is the whole point of the PEP. I want to be able to
*syntactically* tell where the suspension points are in coroutines.

Doesn't "yield from" already do that?


Currently this means looking for yield [from]; PEP 492 just adds looking
for await and async [for|with]. Making await() a function defeats the
purpose because now aliasing can hide its presence, and we're back in
the land of gevent or stackless (where *anything* can potentially
suspend the current task). I don't want to live in that land.
I don't think I was clear enough. I said that "await" *is* a function, 
not that is should be disguised as one. Reading the code, 
"GetAwaitableIter" would be a better name for that element of the 
implementation. It is a straightforward non-blocking function.




On Sun, Apr 26, 2015 at 1:21 PM, Mark Shannon mailto:[email protected]>> wrote:

Hi,

I was looking at PEP 492 and it seems to me that no new syntax is
required.

Looking at the code, it does four things; all of which, or a
functional equivalent, could be done with no new syntax.
1. Make a normal function into a generator or coroutine. This can be
done with a decorator.
2. Support a parallel set of special methods starting with 'a' or
'async'. Why not just use the current set of special methods?
3. "await". "await" is an operator that takes one argument and
produces a single result, without altering flow control and can thus
be replaced by an function.
4. Asynchronous with statement. The PEP lists the equivalent as
"with (yield from xxx)" which doesn't seem so bad.

Please don't add unnecessary new syntax.

Cheers,
Mark.

P.S. I'm not objecting to any of the other new features proposed,
just the new syntax.
___
Python-Dev mailing list
[email protected] 
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org




--
--Guido van Rossum (python.org/~guido )

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492: No new syntax is required

2015-04-27 Thread Mark Shannon 



On 26/04/15 23:24, Nick Coghlan wrote:


On 27 Apr 2015 07:50, "Mark Shannon" mailto:[email protected]>> wrote:
 > On 26/04/15 21:40, Yury Selivanov wrote:
 >>
 >> But it's hard.  Iterating through something asynchronously?  Write a
 >> 'while True' loop.  Instead of 1 line you now have 5 or 6.  Want to
 >> commit your database transaction?  Instead of 'async with' you will
 >> write 'try..except..finally' block, with a very high probability to
 >> introduce a bug, because you don't rollback or commit properly or
 >> propagate exception.
 >
 > I don't see why you can't do transactions using a 'with' statement.

Because you need to pass control back to the event loop from the
*__exit__* method in order to wait for the commit/rollback operation
without blocking the scheduler. The "with (yield from cm())" formulation
doesn't allow either __enter__ *or* __exit__ to suspend the coroutine to
wait for IO, so you have to do the IO up front and return a fully
synchronous (but still non-blocking) CM as the result.

True. The 'with' statement cannot support this use case, but
try-except can do the job:

trans = yield from db_conn.transaction()
try:
...
except:
yield from trans.roll_back()
raise
yield from trans.commit()

Admittedly not as elegant as the 'with' statement, but perfectly readable.



We knew about these problems going into PEP 3156
(http://python-notes.curiousefficiency.org/en/latest/pep_ideas/async_programming.html#using-special-methods-in-explicitly-asynchronous-code)
so it's mainly a matter of having enough experience with asyncio now to
be able to suggest specific syntactic sugar to make the right way and
the easy way the same way.
asyncio is just one module amongst thousands, does it really justify 
special syntax?


Cheers,
Mark.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-27 Thread Greg Ewing 

Yury Selivanov wrote:

I've done some experiments with grammar, and it looks like
we indeed can parse await quite differently from yield. Three
different options:


You don't seem to have tried what I suggested, which is
to make 'await' a unary operator with the same precedence
as '-', i.e. replace

   factor: ('+'|'-'|'~') factor | power

with

   factor: ('+'|'-'|'~'|'await') factor | power

That would allow

  await a()
  res = await a() + await b()
  res = await await a()
  if await a(): pass
  return await a()
  print(await a())
  func(arg=await a())
  await a() * b()

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 492 vs. PEP 3152, new round

2015-04-27 Thread Yury Selivanov 

Hi Greg,

I don't want this: "await a() * b()" to be parsed, it's not meaningful.

Likely you'll see "await await a()" only once in your life, so I'm fine 
to use parens for it (moreover, I think it reads better with parens)


Yury


On 2015-04-27 8:52 AM, Greg Ewing wrote:

Yury Selivanov wrote:

I've done some experiments with grammar, and it looks like
we indeed can parse await quite differently from yield. Three
different options:


You don't seem to have tried what I suggested, which is
to make 'await' a unary operator with the same precedence
as '-', i.e. replace

   factor: ('+'|'-'|'~') factor | power

with

   factor: ('+'|'-'|'~'|'await') factor | power

That would allow

  await a()
  res = await a() + await b()
  res = await await a()
  if await a(): pass
  return await a()
  print(await a())
  func(arg=await a())
  await a() * b()



___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 492: async/await in Python; v3

2015-04-27 Thread Yury Selivanov 

Hi python-dev,

Another round of updates.  Reference implementation
has been updated: https://github.com/1st1/cpython/tree/await
(includes all things from the below summary of updates +
tests).


Summary:


1. "PyTypeObject.tp_await" slot.  Replaces "tp_reserved".
This is to enable implementation of Futures with C API.
Must return an iterator if implemented.


2. New grammar for "await" expressions, see
'Syntax of "await" expression' section


3. inspect.iscoroutine() and inspect.iscoroutineobjects()
functions.


4. Full separation of coroutines and generators.
This is a big one; let's discuss.

a) Coroutine objects raise TypeError (is NotImplementedError
better?) in their __iter__ and __next__.  Therefore it's
not not possible to pass them to iter(), tuple(), next() and
other similar functions that work with iterables.

b) Because of (a), for..in iteration also does not work
on coroutines anymore.

c) 'yield from' only accept coroutine objects from
generators decorated with 'types.coroutine'. That means
that existing asyncio generator-based coroutines will
happily yield from both coroutines and generators.
*But* every generator-based coroutine *must* be
decorated with `asyncio.coroutine()`.  This is
potentially a backwards incompatible change.

d) inspect.isgenerator() and inspect.isgeneratorfunction()
return `False` for coroutine objects & coroutine functions.

e) Should we add a coroutine ABC (for cython etc)?

I, personally, think this is highly necessary. First,
separation of coroutines from generators is extremely
important. One day there won't be generator-based
coroutines, and we want to avoid any kind of confusion.
Second, we only can do this in 3.5.  This kind of
semantics change won't be ever possible.

asyncio recommends using @coroutine decorator, and most
projects that I've seen do use it.  Also there is no
reason for people to use iter() and next() functions
on coroutines when writing asyncio code. I doubt that
this will cause serious backwards compatibility problems
(asyncio also has provisional status).


Thank you,
Yury


PEP: 492
Title: Coroutines with async and await syntax
Version: $Revision$
Last-Modified: $Date$
Author: Yury Selivanov 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 09-Apr-2015
Python-Version: 3.5
Post-History: 17-Apr-2015, 21-Apr-2015, 27-Apr-2015


Abstract


This PEP introduces new syntax for coroutines, asynchronous ``with``
statements and ``for`` loops.  The main motivation behind this proposal
is to streamline writing and maintaining asynchronous code, as well as
to simplify previously hard to implement code patterns.


Rationale and Goals
===

Current Python supports implementing coroutines via generators (PEP
342), further enhanced by the ``yield from`` syntax introduced in PEP
380. This approach has a number of shortcomings:

* it is easy to confuse coroutines with regular generators, since they
  share the same syntax; async libraries often attempt to alleviate
  this by using decorators (e.g. ``@asyncio.coroutine`` [1]_);

* it is not possible to natively define a coroutine which has no
  ``yield`` or  ``yield from`` statements, again requiring the use of
  decorators to fix potential refactoring issues;

* support for asynchronous calls is limited to expressions where
  ``yield`` is allowed syntactically, limiting the usefulness of
  syntactic features, such as ``with`` and ``for`` statements.

This proposal makes coroutines a native Python language feature, and
clearly separates them from generators.  This removes
generator/coroutine ambiguity, and makes it possible to reliably define
coroutines without reliance on a specific library.  This also enables
linters and IDEs to improve static code analysis and refactoring.

Native coroutines and the associated new syntax features make it
possible to define context manager and iteration protocols in
asynchronous terms. As shown later in this proposal, the new ``async
with`` statement lets Python programs perform asynchronous calls when
entering and exiting a runtime context, and the new ``async for``
statement makes it possible to perform asynchronous calls in iterators.


Specification
=

This proposal introduces new syntax and semantics to enhance coroutine
support in Python, it does not change the internal implementation of
coroutines, which are still based on generators.

It is strongly suggested that the reader understands how coroutines are
implemented in Python (PEP 342 and PEP 380).  It is also recommended to
read PEP 3156 (asyncio framework) and PEP 3152 (Cofunctions).

From this point in this document we use the word *coroutine* to refer
to functions declared using the new syntax.  *generator-based
coroutine* is used where necessary to refer to coroutines that are
based on generator syntax.


New Coroutine Declaration Syntax


The following new syntax is used to declare a coroutine::

async def read_data(db

Re: [Python-Dev] PEP 492: async/await in Python; v3

2015-04-27 Thread Stefan Behnel 
Yury Selivanov schrieb am 28.04.2015 um 05:07:
> e) Should we add a coroutine ABC (for cython etc)?

Sounds like the right thing to do, yes. IIUC, a Coroutine would be a new
stand-alone ABC with send, throw and close methods.

Should a Generator then inherit from both Iterator and Coroutine, or would
that counter your intention to separate coroutines from generators as a
concept? I mean, they do share the same interface ...

It seems you're already aware of

https://bugs.python.org/issue24018

Stefan


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com