Re: [Python-ideas] Runtime types vs static types

[rym...@gmail.com]

For some background on the removal of __instancecheck__, check the linked
issues here:


https://github.com/python/typing/issues/135


--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttp://refi64.com

On Jun 25, 2017 at 8:11 AM, > wrote:

On Sat, Jun 24, 2017 at 11:30 PM, Lucas Wiman  wrote:


> ​
> On Sat, Jun 24, 2017 at 12:42 PM, Koos Zevenhoven 
>  wrote:
>
>> There has been some discussion here and there concerning the differences
>> between runtime types and static types (mypy etc.). What I write below is
>> not really an idea or proposal---just a perspective, or a topic that people
>> may want to discuss. Since the discussion on this is currently very fuzzy
>> and scattered and not really happening either AFAICT (I've probably missed
>> many discussions, though). Anyway, I thought I'd give it a shot:
>>
>>
​[...]​



> Regarding runtime types and isinstance, let's look at the Iterable[int]
>> example. For this case, there are a few options:
>>
>> 1) Don't implement isinstance
>>
>> This is problematic for runtime uses of annotations.
>>
>> 2) isinstance([1, '2', 'three'], Iterable[int]) returns True
>>
>> This is in fact now the case. This is ok for many runtime situations, but
>> lacks precision compared to the static version. One may want to distinguish
>> between Iterable[int] and Iterable[str] at runtime (e.g. the multidispatch
>> example above).
>>
>> 3) Check as much as you can at runtime
>>
>> There could be something like Reiterable, which means the object is not
>> consumed by iterating over it, so one could actually check if all elements
>> are instances of int. This would be useful in some situations, but not
>> available for every object. Furthermore, the check could take an arbitrary
>> amount of time so it is not really suitable for things like multidispatch
>> or some matching constructs etc., where the performance overhead of the
>> type check is really important.
>>
>> 4) Do a deeper check than in (2) but trust the annotations
>>
>> For example, an instance of a class that has a method like
>>
>> def __iter__(self) -> Iterator[int]:
>> some code
>>
>> could be identified as Iterable[int] at runtime, even if it is not
>> guaranteed that all elements are really integers.
>>
>> On the other hand, an object returned by
>>
>> def get_ints() -> Iterable[int]:
>> some code
>>
>> does not know its own annotations, so the check is difficult to do at
>> runtime. And of course, there may not be annotations available.
>>
>> 5) Something else?
>>
>>
>> And what about PEP544 (protocols), which is being drafted? The PEP seems
>> to aim for having type objects that represent duck-typing
>> protocols/interfaces. Checking whether a protocol is implemented by an
>> object or type is clearly a useful thing to do at runtime, but it is not
>> really clear if isinstance would be a guaranteed feature for PEP544
>> Protocols.
>>
>> So one question is, is it possible to draw the lines between what works
>> with isinstance and what doesn't, and between what details are checked by
>> isinstance and what aren't? -- Or should insinstance be reserved for a more
>> limited purpose, and add another check function, say `implements(...)`,
>> which would perhaps guarantee some answer for all combinations of object
>> and type?
>>
>
>>
> I'm guessing to implement PEP 544, many of the `__instancecheck__` and
> `__subclasscheck__` methods in `typing.py` would need to be updated to
> check the `__annotations__` of the class of the object it's passed against
> its own definition, (covered in this section
> 
> of the PEP).
>
>
​I may have missed something, but I believe PEP544 is ​not suggesting that
annotations would have any effect on isinstance. Instead, isinstance would
by default not work.



> I've been somewhat surprised that many of the `__instancecheck__`
> implementations do not work at runtime, even when the implementation would
> be trivial (e.g. for `Union`), or would not have subtle edge cases due to
> immutability (e.g. for `Tuple`, which cannot be used for checking
> parameterized instances). This seems like counterintuitive behavior that
> would be straightforward to fix, unless there are subtleties & edge cases
> I'm missing.
>
>
​Tuple is an interesting case, because for small tuples (say 2- or
3-tuples), it makes perfect sense to check the types of all elements for
some runtime purposes.​ Regarding Union, I believe the current situation
has a lot to do with the fact that the relation between type annotations
and runtime behavior hasn't really settled yet.


If people are amenable to updating those cases, I'd be interested in
> submitting a patch to that effect.
>
>
​Thanks for letting us know. (There may not be an instant decision on this
particular case, though, but who knows :)

-- Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
_

Re: [Python-ideas] Allow function to return multiple values

[rym...@gmail.com]

IIRC I'm pretty sure the OP just didn't know about the existence of tuple
unpacking and the ability to use that to return multiple values.


--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttp://refi64.com

On Jun 25, 2017 at 6:09 PM, > wrote:

joannah nanjekye wrote:


> [...]
>
>Today I was writing an example snippet for the book and needed to write a
>function that returns two values something like this:
>
>def return_multiplevalues(num1, num2):
> return num1, num2
>
> I noticed that this actually returns a tuple of the values which I did not
>want in the first place.I wanted python to return two values in their own
>types so I can work with them as they are but here I was stuck with working
>around a tuple.

It was quite puzzling at first what was the actual idea but probably I
can guess why this question came up by you.
It seems to me (I am just intuitively guessing that) that you were about to
write a procedure which operates on global variables.
If so you should use the keyword "global" for that.
E.g. if you want to work with the variables defined in other
part of the code you can simply do it:

x = 0
y = 0
def move():
global x, y
x = x + 10
y = y + 20
move()
print (x,y)

This function will change x and y (global variables in this case).
Note that without the line with the "global" statement this will not work.
Another typical usage is initialising variables inside a procedure:

def init_coordinates():
global x,y
x=0
y=0
init_coordinates()
print (x,y)


So for some reason it seemed to me that you are trying to do
something like that.

>My proposal is we provide a way of functions returning multiple values.
>This has been implemented in languages like Go and I have found many cases
>where I needed and used such a functionality. I wish for this convenience
>in python so that I don't  have to suffer going around a tuple.

So if using globals as in the above examples you certinly don't
have to suffer going around a tuple.



Mikhail
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Python 4: Concatenation

[rym...@gmail.com]

I feel like this would literally break the world for almost no real
benefit...


--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttp://refi64.com

On Jun 29, 2017 at 6:33 PM, > wrote:

Step 1. get rid of + for strings, lists, etc. (string/list concatenation
is not addition)

Step 2. add concatenation operator for strings, lists, and basically
anything that can be iterated. effectively an overloadable
itertools.chain. (list cat list = new list, not iterator, but
effectively makes itertools.chain useless.)

Step 3. add decimal concatenation operator for numbers: 2 cat 3 == 23,
22 cat 33 = 2233, etc. if you need bitwise concatenation, you're already
in bitwise "hack" land so do it yourself. (no idea why bitwise is
considered hacky as I use it all the time, but oh well)

Step 4. make it into python 4, since it breaks backwards compatibility.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Bytecode JIT

[rym...@gmail.com]

This is literally PyPy. There's little reason for something like this to
end up in official CPython, at least for now.


--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttp://refi64.com

On Jul 1, 2017 at 5:53 PM, > wrote:



On 2017-07-01 07:34 PM, Victor Stinner wrote:
> Let's say that you have a function "def mysum (x; y): return x+y", do
> you always want to use your new IADD instruction here? What if I call
> mysum ("a", "b")?
>
> Victor

Let's say that you do. Given how short it is, it would just get inlined.
Your call of mysum ("a", "b") would indeed not use IADD, nor would it be
a call. It would potentially not invoke any operators, but instead get
replaced with "ab".

When you have a tracing JIT, you can do away with a lot of overhead. You
can inline functions, variables, do away with typechecks, and many other
things. This holds true even if that JIT never emits a single byte of
machine code.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] New PEP 550: Execution Context

[rym...@gmail.com]

So, I'm hardly an expert when it comes to things like this, but there are
two things about this that don't seem right to me. (Also, I'd love to
respond inline, but that's kind of difficult from a mobile phone.)

The first is how set/get_execution_context_item take strings. Inevitably,
people are going to do things like:

CONTEXT_ITEM_NAME = 'foo-bar'
...
sys.set_execution_context_item(CONTEXT_ITEM_NAME, 'stuff')

IMO it would be nicer if there could be a key object used instead, e.g.

my_key = sys.execution_context_key('name-here-for-debugging-purposes')
sys.set_execution_context_item(my_key, 'stuff')

The advantage here would be no need for string constants and no potential
naming conflicts (the string passed to the key creator would be used just
for debugging, kind of like Thread names).


Second thing is this:

def context(x):
old_x = get_execution_context_item('x')
set_execution_context_item('x', x)
try:
yield
finally:
set_execution_context_item('x', old_x)



If this would be done frequently, a context manager would be a *lot* more
Pythonic, e.g.:

with sys.temp_change_execution_context('x', new_x):
# ...

--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttp://refi64.com

On Aug 11, 2017 at 5:38 PM, >
wrote:

Hi,

This is a new PEP to implement Execution Contexts in Python.

The PEP is in-flight to python.org, and in the meanwhile can
be read on GitHub:

https://github.com/python/peps/blob/master/pep-0550.rst

(it contains a few diagrams and charts, so please read it there.)

Thank you!
Yury


PEP: 550
Title: Execution Context
Version: $Revision$
Last-Modified: $Date$
Author: Yury Selivanov 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 11-Aug-2017
Python-Version: 3.7
Post-History: 11-Aug-2017


Abstract


This PEP proposes a new mechanism to manage execution state--the
logical environment in which a function, a thread, a generator,
or a coroutine executes in.

A few examples of where having a reliable state storage is required:

* Context managers like decimal contexts, ``numpy.errstate``,
  and ``warnings.catch_warnings``;

* Storing request-related data such as security tokens and request
  data in web applications;

* Profiling, tracing, and logging in complex and large code bases.

The usual solution for storing state is to use a Thread-local Storage
(TLS), implemented in the standard library as ``threading.local()``.
Unfortunately, TLS does not work for isolating state of generators or
asynchronous code because such code shares a single thread.


Rationale
=

Traditionally a Thread-local Storage (TLS) is used for storing the
state.  However, the major flaw of using the TLS is that it works only
for multi-threaded code.  It is not possible to reliably contain the
state within a generator or a coroutine.  For example, consider
the following generator::

def calculate(precision, ...):
with decimal.localcontext() as ctx:
# Set the precision for decimal calculations
# inside this block
ctx.prec = precision

yield calculate_something()
yield calculate_something_else()

Decimal context is using a TLS to store the state, and because TLS is
not aware of generators, the state can leak.  The above code will
not work correctly, if a user iterates over the ``calculate()``
generator with different precisions in parallel::

g1 = calculate(100)
g2 = calculate(50)

items = list(zip(g1, g2))

# items[0] will be a tuple of:
#   first value from g1 calculated with 100 precision,
#   first value from g2 calculated with 50 precision.
#
# items[1] will be a tuple of:
#   second value from g1 calculated with 50 precision,
#   second value from g2 calculated with 50 precision.

An even scarier example would be using decimals to represent money
in an async/await application: decimal calculations can suddenly
lose precision in the middle of processing a request.  Currently,
bugs like this are extremely hard to find and fix.

Another common need for web applications is to have access to the
current request object, or security context, or, simply, the request
URL for logging or submitting performance tracing data::

async def handle_http_request(request):
context.current_http_request = request

await ...
# Invoke your framework code, render templates,
# make DB queries, etc, and use the global
# 'current_http_request' in that code.

# This isn't currently possible to do reliably
# in asyncio out of the box.

These examples are just a few out of many, where a reliable way to
store context data is absolutely needed.

The inability to use TLS for asynchronous code has lead to
proliferation of ad-hoc solutions, limited to be supported only by
code that was explicitly enabled to work with them.

Current status quo is that any library, including th

Re: [Python-ideas] Please consider adding context manager versions of setUp/tearDown to unittest.TestCase

[rym...@gmail.com]

TBH you're completely right. Every time I see someone using unittest
andItsHorriblyUnpythonicNames, I want to kill a camel.

Sometimes, though, I feel like part of the struggle is the alternative. If
you dislike unittest, but pytest is too "magical" for you, what do you use?
Many Python testing tools like nose are just test *runners*, so you still
need something else. In the end, many just end up back at unittest, maybe
with nose on top.

As much as I hate JavaScript, their testing libraries are leagues above
what Python has.

--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttp://refi64.com

On Aug 22, 2017 at 5:09 PM, > wrote:

** Caution: cranky curmudgeonly opinionated comment ahead: **


unitest is such an ugly Java-esque static mess of an API that there's
really no point in trying to clean it up and make it more pythonic -- go
off and use pytest and be happier.

-CHB



On Tue, Aug 22, 2017 at 5:42 AM, Nick Coghlan  wrote:

> On 22 August 2017 at 15:34, Nick Coghlan  wrote:
> > On 21 August 2017 at 11:32, Neil Girdhar  wrote:
> >> This question describes an example of the problem:
> >> https://stackoverflow.com/questions/8416208/in-python-
> is-there-a-good-idiom-for-using-context-managers-in-setup-teardown.
> >> You want to invoke a context manager in your setup/tearing-down, but the
> >> easiest way to do that is to override run, which seems ugly.
> >
> > Using context managers when you can't use a with statement is one of
> > the main use cases for contextlib.ExitStack():
> >
> > def setUp(self):
> > self._resource_stack = stack = contextlib.ExitStack()
> > self._resource = stack.enter_context(MyResource())
> >
> > def tearDown(self):
> > self._resource_stack.close()
> >
> > I posted that as an additional answer to the question:
> > https://stackoverflow.com/questions/8416208/in-python-
> is-there-a-good-idiom-for-using-context-managers-in-
> setup-teardown/45809502#45809502
>
> Sjoerd pointed out off-list that this doesn't cover the case where
> you're acquiring multiple resources and one of the later acquisitions
> fails, so I added the ExitStack idiom that covers that case (using
> stack.pop_all() as the last operation in a with statement):
>
> def setUp(self):
> with contextlib.ExitStack() as stack:
> self._resource1 = stack.enter_context(GetResource())
> self._resource2 = stack.enter_context(GetOtherResource())
> # Failures before here -> immediate cleanup
> self.addCleanup(stack.pop_all().close)
> # Now cleanup won't happen until the cleanup functions run
>
> I also remember that using addCleanup lets you avoid defining tearDown
> entirely.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___ Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct:
http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Extension of python/json syntax to support explicitly sets and ordered dict.

[rym...@gmail.com]

IIRC in CPython 3.6 and PyPy dicts are ordered based on insertion anyway;
although it's an implementation-specific detail, realistically it removes
some of the use cases for ordered dictionary literals.


--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttp://refi64.com

On Sep 7, 2017 at 6:40 AM, >
wrote:

Hi all,

few days ago I thought about a way to rapresent sets and
ordered dicts using a json compatible syntax, these are my conclusions:

A set could be defined as { item1, item2, item3[...] }
with {,} as an empty set

An ordered dict could be defined as [ item1: value1, item2: value2 ... ]
with [:] ase an empty odered dict

It could be used inside python code or to serialize python structures in
a json-like format (pyson maybe ?).

What do you think about ?

  Kind regards, Matteo.

--
  email:   nast...@alternativeoutput.it,   matteo.nast...@gmail.com
  web:   www.alternativeoutput.it   irc: #linux...@irc.freenode.net
 linkedin: http://lnkd.in/SPQG87
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Coming up with an alternative to PEP 505's None-aware operators

[rym...@gmail.com]

I don't know...to me this looks downright ugly and an awkward special case.
It feels like it combines reading difficulty of inline assignment with the
awkwardness of a magic word and the ugliness of using ?. Basically, every
con of the other proposals combined...

--
Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone
elsehttps://refi64.com/


On Feb 15, 2018 at 8:07 PM, > wrote:

The recent thread on variable assignment in comprehensions has
prompted me to finally share
https://gist.github.com/ncoghlan/a1b0482fc1ee3c3a11fc7ae64833a315 with
a wider audience (see the comments there for some notes on iterations
I've already been through on the idea).

== The general idea ==

The general idea would be to introduce a *single* statement local
reference using a new keyword with a symbolic prefix: "?it"

* `(?it=expr)` is a new atomic expression for an "it reference
binding" (whitespace would be permitted around "?it" and "=", but PEP
8 would recommend against it in general)
* subsequent subexpressions (in execution order) can reference the
bound subexpression using `?it` (an "it reference")
* `?it` is reset between statements, including before entering the
suite within a compound statement (if you want a persistent binding,
use a named variable)
* for conditional expressions, put the reference binding in the
conditional, as that gets executed first
* to avoid ambiguity, especially in function calls (where it could be
confused with keyword argument syntax), the parentheses around
reference bindings are always required
* unlike regular variables, you can't close over statement local
references (the nested scope will get an UnboundLocalError if you try
it)

The core inspiration here is English pronouns (hence the choice of
keyword): we don't generally define arbitrary terms in the middle of
sentences, but we *do* use pronouns to refer back to concepts
introduced earlier in the sentence. And while it's not an especially
common practice, pronouns are sometimes even used in a sentence
*before* the concept they refer to ;)

If we did pursue this, then PEPs 505, 532, and 535 would all be
withdrawn or rejected (with the direction being to use an it-reference
instead).

== Examples ==

`None`-aware attribute access:

value = ?it.strip()[4:].upper() if (?it=var1) is not None else None

`None`-aware subscript access:

value = ?it[4:].upper() if (?it=var1) is not None else None

`None`-coalescense:

value = ?it if (?it=var1) is not None else ?it if (?it=var2) is
not None else var3

`NaN`-coalescence:

value = ?it if not math.isnan((?it=var1)) else ?it if not
math.isnan((?that=var2)) else var3

Conditional function call:

value = ?it() if (?it=calculate) is not None else default

Avoiding repeated evaluation of a comprehension filter condition:

filtered_values = [?it for x in keys if (?it=get_value(x)) is not None]

Avoiding repeated evaluation for range and slice bounds:

range((?it=calculate_start()), ?it+10)
data[(?it=calculate_start()):?it+10]

Avoiding repeated evaluation in chained comparisons:

value if (?it=lower_bound()) <= value < ?it+tolerance else 0

Avoiding repeated evaluation in an f-string:

print(f"{?it=get_value()!r} is printed in pure ASCII as {?it!a}
and in Unicode as {?it}"

== Possible future extensions ==

One possible future extension would be to pursue PEP 3150, treating
the nested namespace as an it reference binding, giving:

sorted_data = sorted(data, key=?it.sort_key) given ?it=:
def sort_key(item):
return item.attr1, item.attr2

(A potential bonus of that spelling is that it may be possible to make
"given ?it=:" the syntactic keyword introducing the suite, allowing
"given" itself to continue to be used as a variable name)

Another possible extension would be to combine it references with `as`
clauses on if statements and while loops:

if (?it=pattern.match(data)) is not None as matched:
...

while (?it=pattern.match(data)) is not None as matched:
...

== Why not arbitrary embedded assignments? ==

Primarily because embedded assignments are inherently hard to read,
especially in long expressions. Restricting things to one pronoun, and
then pursuing PEP 3150's given clause in order to expand to multiple
statement local names should help nudge folks towards breaking things
up into multiple statements rather than writing ever more complex
one-liners.

That said, the ?-prefix notation is deliberately designed such that it
*could* be used with arbitrary identifiers rather then being limited
to a single specific keyword, and the explicit lack of closure support
means that there wouldn't be any complex nested scope issues
associated with lambda expressions, generator expressions, or
container comprehensions.

With that approach, "?it" would just be an idiomatic default name like
"self" or "cls" rather than being a true keyword. Given arbitrary
identifier support, some of the earlier exampl