Re: a *= b not equivalent to a = a*b

[mlzarathustra]

Yes, I just worked that out. It's the integer math that's the problem.

I guess this has been fixed in python 3, but unfortunately it seems that most 
people are still using python 2. 

Thanks for all the help!

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[mlzarathustra]

I was being facetious, but behind it is a serious point. Neither the APL nor 
the J languages use precedence even though their inventor, Ken Iverson, was a 
mathematician.

That was to support functional programming dating back to the 1970's.

However, precedence wasn't the problem in this case, it was the type conversion.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[Jussi Piitulainen]

mlzarathus...@gmail.com writes:

> Yes, I just worked that out. It's the integer math that's the problem.
>
> I guess this has been fixed in python 3 [- -]

Note that division in Python 3 produces approximate results (floating
point numbers). This may or may not be what you want in this exercise.
I would blame this problem entirely on the precedence issue and just do
the multiplication first. The algorithm is pretty neat that way.

(Meta) Also, please leave some relevant context so it's easier to follow
the discussion. It's a general principle, and particularly acute in
comp.lang.python where many messages fail to identify their parent.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[mlzarathustra]

Here's the key:

$ python2
Python 2.7.10 ...
>>> 1/2
0
>>>

$ python
Python 3.5.1 ...
>>> 1/2
0.5
>>> 1//2
0
>>>

I read about this awhile ago, but it's not until it bites you that you remember 
fully.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[Erik]

On 26/08/16 08:44, mlzarathus...@gmail.com wrote:

Here's the key:

$ python2
Python 2.7.10 ...

1/2

0




$ python
Python 3.5.1 ...

1/2

0.5

1//2

0




I read about this awhile ago, but it's not until it bites you that you remember 
fully.


How is this related to your question? The example explicitly says Python 
2 and doesn't use the '//' operator.


E.
--
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[Erik]

On 26/08/16 08:14, mlzarathus...@gmail.com wrote:


I was being facetious, but behind it is a serious point. Neither the APL nor 
the J languages use precedence even though their inventor, Ken Iverson, was a 
mathematician.

That was to support functional programming dating back to the 1970's.


Precedence is not the issue here anyway. Both '*' and '/' have the same 
precedence. The two operations in your expressions have to be evaluated 
in SOME order - either left-to-right or right-to-left.



The issue is twofold:

Firstly, the compound assignments will always evaluate their RHS before 
performing the assignment - this is not the same as operator precedence 
- they have to, else what does Python pass to the function that knows 
what the compound assignment means for that type (e.g. sequences may be 
extended for '*=')? So for compound assignments there is always 
effectively an implicit set of parentheses around the RHS compared to 
the expression without the assignment operator (if you think of the 
compound assignment as being a _part_ of the overall expression).


Secondly, the way you have chosen to layout your code has fooled your 
brain into thinking that the division is performed before the 
multiplication. Python doesn't care that you jammed the sub-expression 
together with no whitespace ;)


E.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Holding until next value change

[Arshpreet Singh]

On Saturday, 20 August 2016 11:38:03 UTC+5:30, Steve D'Aprano  wrote:

> state = ignore_negative  # DON'T call the function yet
> for value in main_call():
> print(value)  # for testing
> if state(value):
> print("changing state")
> state = TABLE[state]

Above code works at some extent but after few minutes of running it
returns and exists by printing 'None' on screen. Let me tell that
main_call() generator spits out values like one more each second after
going through some sort of heavy calculations(but even 50% of RAM is
available) it exits by printing 'None' on screen. Is it some kind of recursion 
problem?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[Christian Gollwitzer]

Am 26.08.16 um 09:53 schrieb Erik:

On 26/08/16 08:44, mlzarathus...@gmail.com wrote:

Here's the key:

$ python2
Python 2.7.10 ...

1/2

0




$ python
Python 3.5.1 ...

1/2

0.5

1//2

0




I read about this awhile ago, but it's not until it bites you that you
remember fully.


How is this related to your question? The example explicitly says Python
2 and doesn't use the '//' operator.



It's related by the fact that a*b/c performs integer division (intended 
by the OP) which gives a different result than a*(b/c) (unintended by 
the OP). Floating point (as in Python 3) *also* may give a different 
result, but the deviation from the "true", i.e. mathematical value, is 
far less than with integer arithmetics.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Python 3 raising an error where Python 2 did not

[d...@forestfield.co.uk]

In a program I'm converting to Python 3 I'm examining a list of divisor values, 
some of which can be None, to find the first with a value greater than 1.

Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> None > 1
False

Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 20:20:57) [MSC v.1600 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> None > 1
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unorderable types: NoneType() > int()

I can live with that but I'm curious why it was decided that this should now 
raise an error.

David Hughes
Forestfield Software
-- 
https://mail.python.org/mailman/listinfo/python-list

PEP 492: isn't the "await" redundant?

[Kouli]

Hello,

recently, I have discovered Python's coroutines and enjoyed the whole
asyncio system a lot. But I ask you to make me understand one thing in
Python's coroutines design: why do we have to use "await" (or "yield
from") in coroutines? Why can coroutines etc. not be used
_from_coroutines_ (designated by 'async def') by a simple call-like
syntax (i.e. without the 'await' keyword)? The same for "await with"
and "await from". At most places a coroutine is referenced from
another coroutine, it is referenced using "await". Couldn't it be
avoided at theese places? This way, one would not have to
differentiate between function and coroutine "call" from within a
coroutine...

Current syntax:

async def work(x):
await asyncio.sleep(x)
def main(x):
 loop.run_until_complete(work(x))

Proposed syntax:

async def work(x):
asyncio.sleep(x) # compiler "adds" 'await' automatically when
in 'async def'
def main(x):
loop.run_until_complete(work(x)) # compiler leaves as is when in 'def'

Historically, generators were defined by using keyword 'yield' inside
its definition. We now have explicit syntax with keyword 'async' so
why should we use yet the additional keyword 'await'? I tried to show
(a minor) example which would need the "leave as is" behavior inside
'async def', but I haven't found such a coroutine refence in examples.
Should it be needed (please, tell me), then it requires a special
syntax (at least for arguments - without arguments one can leave out
the parentheses).

Kouli
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 raising an error where Python 2 did not

[Chris Angelico]

On Fri, Aug 26, 2016 at 6:50 PM, d...@forestfield.co.uk
 wrote:
> In a program I'm converting to Python 3 I'm examining a list of divisor 
> values, some of which can be None, to find the first with a value greater 
> than 1.
>
> Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on 
> win32
> Type "help", "copyright", "credits" or "license" for more information.
 None > 1
> False
>
> Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 20:20:57) [MSC v.1600 64 bit 
> (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
 None > 1
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: unorderable types: NoneType() > int()
>
> I can live with that but I'm curious why it was decided that this should now 
> raise an error.

Because it doesn't make sense to compare these things in this way. You
can compare some things (eg integers and floats), but others don't
usefully arrange themselves into any sort of order. It tends to lead
to odd situations:

rosuav@sikorsky:~$ python2
Python 2.7.12+ (default, Aug  4 2016, 20:04:34)
[GCC 6.1.1 20160724] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 10 < "15"
True
>>> "15" < 20
False
>>> "asdf" < (1,2,3)
True
>>> "asdf" > [1,2,3]
True
>>>
rosuav@sikorsky:~$ python3
Python 3.6.0a4+ (default:4b64a049f451+, Aug 19 2016, 23:41:43)
[GCC 6.1.1 20160802] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 10 < "15"
Traceback (most recent call last):
  File "", line 1, in 
TypeError: '<' not supported between instances of 'int' and 'str'
>>>

With Py3, it's simple and obvious: comparing integers and strings is
an error. With Py2, there's the extremely odd rule that all integers
are lower than all lists are lower than all strings are lower than all
tuples - it's based on the name of the type. (Except for 'long', which
appears to sort as though it were called 'int'. I think.) The Py2
behaviour allows you to get a consistent-but-meaningless sort order
among disparate types, but it's a bug magnet, and it goes against
Python's philosophy of telling you about problems right away. Py3
fixed quite a number of those kinds of issues (eg you can't combine
byte strings and Unicode strings in Py3, yet you can in Py2), with the
result that a number of data-dependent bugs in Py2 become instant
exceptions in Py3.

The simplest way to fix your code here is to explicitly provide a
default. For instance:

divisors = [None, 1, None, 2, 7, None, 1]
greater_than_one = (div for div in divisors if (div or 0) > 1)

The (div or 0) part means that any None will be treated as zero.
(Also, zero will be treated as zero - be careful of this if you change
the default, eg to 1.) At that point, all your comparisons will
involve numbers, which have well-defined inequality comparisons.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[Peter Otten]

mlzarathus...@gmail.com wrote:

> Yes, I just worked that out. It's the integer math that's the problem.
> 
> I guess this has been fixed in python 3, but unfortunately it seems that
> most people are still using python 2.

Note that you can get Python 3's default behaviour in Python 2 with

from __future__ import division

at the beginning of the module.

>>> 2/3
0
>>> from __future__ import division
>>> 2/3
0.



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 492: isn't the "await" redundant?

[Chris Angelico]

On Fri, Aug 26, 2016 at 6:55 PM, Kouli  wrote:
> recently, I have discovered Python's coroutines and enjoyed the whole
> asyncio system a lot. But I ask you to make me understand one thing in
> Python's coroutines design: why do we have to use "await" (or "yield
> from") in coroutines? Why can coroutines etc. not be used
> _from_coroutines_ (designated by 'async def') by a simple call-like
> syntax (i.e. without the 'await' keyword)?

Two reasons. One is that Python allows you to call any function and
inspect its return value - async functions are no different, and they
do return something. The other is that it makes yield points obvious.
Consider this hypothetical function:

async def get_user_data(id):
await db.query("select name from users where id=?", (id,))
name = await db.fetchone()[0]
# transaction handling elided
return name

Now, suppose we're trying to figure out what's going on. One good
solid technique is what I call "IIDPIO debugging": If In Doubt, Print
It Out.

async def get_user_data(id):
print("Starting g_u_d")
q = db.query("select name from users where id=?", (id,))
print(q)
await q
f = db.fetchone()[0]
print(f)
name = await f
# transaction handling still elided
print("g_u_d: Returning %r" % name)
return name

It's completely obvious, here, that this function will call db.query,
print stuff out, and then put itself on ice until the query's done,
before attempting the fetch. If the call on the second line
automatically put the function into waiting mode, this display would
be impossible, and the wait points would be entirely implicit. (If you
want implicit wait points, use threading, not async I/O.)

It's also possible to wait for things that didn't come from function
calls per se. For instance, the same database lookup could be
implemented using an ORM, something like this:

class Table:
async def __getitem__(self, id):
if id in self._cache:
return self._cache[id]
await db.query(...)
data = await db.fetchone()
self._cache[id] = self.make_object(...)
return self._cache[id]

Whether this is good code or not is up to you, but it's perfectly
legal, and would be used as "users = Table(...); my_user = await
users[123]". To allow that, Python absolutely has to allow arbitrary
expressions to be waited on, not just function calls.

Does that answer the question?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 raising an error where Python 2 did not

[Peter Otten]

d...@forestfield.co.uk wrote:

> In a program I'm converting to Python 3 I'm examining a list of divisor
> values, some of which can be None, to find the first with a value greater
> than 1.
> 
> Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)]
> on win32 Type "help", "copyright", "credits" or "license" for more
> information.
 None > 1
> False
> 
> Python 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 20:20:57) [MSC v.1600 64
> bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for
> more information.
 None > 1
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: unorderable types: NoneType() > int()
> 
> I can live with that but I'm curious why it was decided that this should
> now raise an error.

Because allowing comparisons between object of arbitrary type does more harm 
than benefit?

[Python 2]
>>> sorted([1, "2", 3])
[1, 3, '2']

In the rare case where you actually want to compare different types you can 
make that explicit:

[Python 3]
>>> sorted([1, "2", 3], key=int)
[1, '2', 3]
>>> sorted([1, "2", 3], key=lambda x: (type(x).__name__, x))
[1, 3, '2']
>>> sorted([1, "2", 3], key=lambda x: (not isinstance(x, str), x))
['2', 1, 3]
>>> from itertools import product
>>> for x, y in product([None, 1], repeat=2):
... print(x, ">", y, "-->", (x is not None, x) > (y is not None, y))
... 
None > None --> False
None > 1 --> False
1 > None --> True
1 > 1 --> False


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 492: isn't the "await" redundant?

[Marko Rauhamaa]

Kouli :
> We now have explicit syntax with keyword 'async' so why should we use
> yet the additional keyword 'await'?

This is an important question.

> This way, one would not have to differentiate between function and
> coroutine "call" from within a coroutine...

You'd still need to remember to add the 'async' keyword all over the
place.

How about making *every* function *always* and async, unconditionally?
That way *every* function would be an async and every function call
would be an await.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 492: isn't the "await" redundant?

[Chris Angelico]

On Fri, Aug 26, 2016 at 7:41 PM, Marko Rauhamaa  wrote:
> How about making *every* function *always* and async, unconditionally?
> That way *every* function would be an async and every function call
> would be an await.

If you want threading, you know where to find it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[BartC]

On 26/08/2016 08:14, mlzarathus...@gmail.com wrote:


However, precedence wasn't the problem in this case, it was the type conversion.


I think it was. I was puzzled as well.

But apparently if you have:

  x * = expr

That's like:

  x = x * (expr)# note the parentheses

which may not always be the same as:

  x = x * expr

as the latter may depends on how operators within expr relate to the 
'*'. In your example, "/" within expr has the same precedence as "*" is 
the "*" is done first not last.


The type conversion may be another issue, but it doesn't explain why the 
results with the /same version/ of Python are different.


--
Bartc




--
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[mlz]

Aha. That's interesting.


On Friday, August 26, 2016 at 2:11:32 AM UTC-7, Peter Otten wrote:
> mlz wrote:
> 
> > Yes, I just worked that out. It's the integer math that's the problem.
> > 
> > I guess this has been fixed in python 3, but unfortunately it seems that
> > most people are still using python 2.
> 
> Note that you can get Python 3's default behaviour in Python 2 with
> 
> from __future__ import division
> 
> at the beginning of the module.
> 
> >>> 2/3
> 0
> >>> from __future__ import division
> >>> 2/3
> 0.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 492: isn't the "await" redundant?

[Marko Rauhamaa]

Chris Angelico :
> On Fri, Aug 26, 2016 at 7:41 PM, Marko Rauhamaa  wrote:
>> How about making *every* function *always* and async,
>> unconditionally? That way *every* function would be an async and
>> every function call would be an await.
>
> If you want threading, you know where to find it.

Ultimately, asyncio and multithreading might well merge. It will be
difficult for a programmer to decide in the beginning of the design
which way to go as the programming models are almost identical.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[mlz]

Partly it's the layout, but mathematically speaking the two should be equal.

(a*b)/c should equal a*(b/c)

The fact that they're not is surprising, because python 2 so seamlessly 
supports big integers in a mathematically correct way.

I consider such surprising behavior to be abstraction leak, which is probably 
why it has been fixed in python 3.

  -= m =-


On Friday, August 26, 2016 at 12:54:02 AM UTC-7, Erik wrote:
> On 26/08/16 08:14, mlz wrote:
> >
> > I was being facetious, but behind it is a serious point. Neither the APL 
> > nor the J languages use precedence even though their inventor, Ken Iverson, 
> > was a mathematician.
> >
> > That was to support functional programming dating back to the 1970's.
> 
> Precedence is not the issue here anyway. Both '*' and '/' have the same 
> precedence. The two operations in your expressions have to be evaluated 
> in SOME order - either left-to-right or right-to-left.
> 
> 
> The issue is twofold:
> 
> Firstly, the compound assignments will always evaluate their RHS before 
> performing the assignment - this is not the same as operator precedence 
> - they have to, else what does Python pass to the function that knows 
> what the compound assignment means for that type (e.g. sequences may be 
> extended for '*=')? So for compound assignments there is always 
> effectively an implicit set of parentheses around the RHS compared to 
> the expression without the assignment operator (if you think of the 
> compound assignment as being a _part_ of the overall expression).
> 
> Secondly, the way you have chosen to layout your code has fooled your 
> brain into thinking that the division is performed before the 
> multiplication. Python doesn't care that you jammed the sub-expression 
> together with no whitespace ;)
> 
> E.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[mlz]

It's true that a*(b/c) yields fractions which would probably accrue accuracy 
errors depending on how those values are implemented. For example, it is 
possible to represent 1/3 internally as two numbers, numerator and denominator, 
thus avoiding the repeating decimal (or binimal, or whatever it's called). I 
believe there are languages that preserve exact accuracy in this way for 
rational fractions. I don't know if Python is one of them.

On the other hand, (a*b)/c is safer since in this case (for the binomial 
coefficient) it always yields an integer.

  -= m =-



On Friday, August 26, 2016 at 1:42:31 AM UTC-7, Christian Gollwitzer wrote:
> Am 26.08.16 um 09:53 schrieb Erik:
> > On 26/08/16 08:44, mlz wrote:
> >> Here's the key:
> >>
> >> $ python2
> >> Python 2.7.10 ...
> > 1/2
> >> 0
> >
> >>
> >> $ python
> >> Python 3.5.1 ...
> > 1/2
> >> 0.5
> > 1//2
> >> 0
> >
> >>
> >> I read about this awhile ago, but it's not until it bites you that you
> >> remember fully.
> >
> > How is this related to your question? The example explicitly says Python
> > 2 and doesn't use the '//' operator.
> >
> 
> It's related by the fact that a*b/c performs integer division (intended 
> by the OP) which gives a different result than a*(b/c) (unintended by 
> the OP). Floating point (as in Python 3) *also* may give a different 
> result, but the deviation from the "true", i.e. mathematical value, is 
> far less than with integer arithmetics.
> 
>   Christian

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to XML?

[Joonas Liik]

On 26 August 2016 at 08:22, Frank Millman  wrote:
> "Peter Otten"  wrote in message news:npn25e$s5n$1...@blaine.gmane.org...
>
> Frank Millman wrote:
>
>>> As you have to keep the "<", why bother?
>>
>>
>> If you mean why don't I convert the '<' to '<', the answer is that I do
>> - I just omitted to say so. However, explicit is better than implicit :-)
>
>
>> Doesn't that make the XML document invalid or changes it in an
>> irreversible way? How would you know whether
>
>
> ""
>
> started out as
>
> ""
>
> or
>
> ""
>
> ?
>
> I cheat ;-)
>
> It is *my* XML, and I know that I only use the offending characters inside
> attributes, and attributes are the only place where double-quote marks are
> allowed.
>
> So this is my conversion routine -
>
> lines = string.split('"')  # split on attributes
> for pos, line in enumerate(lines):
>if pos%2:  # every 2nd line is an attribute
>lines[pos] = line.replace('<', '<').replace('>', '>')
> return '"'.join(lines)
>
> Frank
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

or.. you could just escape all & as & before escaping the > and <,
and do the reverse on decode
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[Jussi Piitulainen]

mlz writes:

> It's true that a*(b/c) yields fractions which would probably accrue
> accuracy errors depending on how those values are implemented. For
> example, it is possible to represent 1/3 internally as two numbers,
> numerator and denominator, thus avoiding the repeating decimal (or
> binimal, or whatever it's called). I believe there are languages that
> preserve exact accuracy in this way for rational fractions. I don't
> know if Python is one of them.

Python doesn't and isn't. Exact fractions are available in the standard
library, but the notation 1/3 gives a floating point approximation.

Floating point arithmetic is usually supported directly in the hardware
in bounded space (64 bits per number). Exact rationals can acquire huge
numerators and denominators surprisingly fast, slowing the computations
down. It's a tradeoff. It's good to know about both.

> On the other hand, (a*b)/c is safer since in this case (for the
> binomial coefficient) it always yields an integer.

When you compute them in that specific way, walking the numerator down
and denominator up.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 492: isn't the "await" redundant?

[Chris Angelico]

(Did you mean to send this to the list? I hope so; I'm replying to the list.)

On Fri, Aug 26, 2016 at 8:30 PM, Milan Krčmář  wrote:
>> Two reasons. One is that Python allows you to call any function and
>> inspect its return value - async functions are no different, and they
>> do return something. The other is that it makes yield points obvious.
>> Consider this hypothetical function:
>
> The return value could be (with proposed syntax) inspected as well.
> The yield point is often visible just from the function you are using:
> async.sleep() vs. sleep() etc.

Not sure how it could be inspected - only the resulting value could.
You couldn't see the Awaitable that gets returned in between.
Depending on the situation, that could be extremely useful.

>> Now, suppose we're trying to figure out what's going on. One good
>> solid technique is what I call "IIDPIO debugging": If In Doubt, Print
>> It Out.
>
> Yes, the 'q = ...; print(q); await q' is a use case to introduce await.
>
>>
>> async def get_user_data(id):
>> print("Starting g_u_d")
>> q = db.query("select name from users where id=?", (id,))
>> print(q)
>> await q
>
>> "users = Table(...); my_user = await users[123]"
>
> An interesting example. But the 'my_user = await users[123]' must have
> appeared inside 'async def',
> so would be written as 'my_user = users[123]' in "my" syntax...

So what that really means is that, the instant something hits an
awaitable, the entire thread gets paused. That's a perfectly
reasonable way of thinking... if you have a concept of threads that
get explicitly spun off. Otherwise, it's a bit tricky, because there's
no easy way to implement the boundary - the point at which the
awaitable gets added to a queue somewhere (ie the top of the event
loop).

> Chris, thank you for such a long reply. I feel being much more
> reconciled with the "verbose" syntax ;-)

No probs, happy to help out.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: a *= b not equivalent to a = a*b

[Chris Angelico]

On Fri, Aug 26, 2016 at 8:20 PM, mlz  wrote:
> I believe there are languages that preserve exact accuracy in this way for 
> rational fractions. I don't know if Python is one of them.

It is, but only if you explicitly request it (due to the performance
impact). Just import it:

#!/usr/bin/python2
import fractions

import sys
FAIL= True if len(sys.argv)>1 else False

def bin(n,k):
rs=1
k=min(k,n-k)
n = fractions.Fraction(n)

for i in range(1,k+1):
if FAIL: rs *= (n-(i-1))/i  # these should be the same,
else: rs = rs * (n-(i-1))/i  #  but apparently are not
return rs


for n in range(10):
for k in range(n+1):
print bin(n,k),
print''


That's the only change you need. The arithmetic will then be done with
ratios of integers, and it'll be exact.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

datetime, time zone and xs:datetime

[Nagy László Zsolt]

  Hello,

I'm having problems finding the "preferrably one" way to convert a
datetime instance into an xs:datetime string.

Here is an example datetime instance with is format:

dt = datetime.now()
print(dt.isoformat()) # prints "2016-08-26 12:41:13.426081+02:00"
print(dt.strftime('%Y-%m-%dT%H:%M:%S%z')) # prints
"2016-08-26T12:41:13+0200"

The main flaw is that the xs:datetime format requires a colon. (See
http://books.xmlschemata.org/relaxng/ch19-77049.html )

The %z format does not contain that. I'm not sure when and how the %z
could be used out of the box, but probably there are datetime formats
when they are needed.

Of course, I can do a workaround:

def datetime_to_xsd_timestamp(dt):
s = dt.strftime('%Y-%m-%dT%H:%M:%S')
tzs = dt.strftime("%z")
if tzs:
s += tzs[:3] + ":" + tzs[3:]
return s

But this does not seem to be the right way to do it. Could we have
datetime.xsdformat()? Or maybe extra format characters in strftime that
represent the hour and the minute part of the time zone offset? Or maybe
I'm not aware of some method in the standard library that converts to
this format back and forth?

Thanks,

   Laszlo



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[Jon Ribbens]

On 2016-08-24, Chris Angelico  wrote:
> On Thu, Aug 25, 2016 at 7:00 AM, eryk sun  wrote:
>> I discovered why "Logs/con.txt" isn't working right in Windows 7,
>> while "Logs/nul.txt" does get redirected correctly to r"\\.\nul".
>> Prior to Windows 8 the console doesn't use an NT device, so the base
>> API has a function named BaseIsThisAConsoleName that looks for names
>> such as r"\\.CON", r"\\.CONIN$", "CON", or r"C:\Temp\con.txt" and
>> returns either "CONIN$" or "CONOUT$" if there's a match. A match for
>> just "CON" maps to one or the other of the latter depending on whether
>> read or write access is desired.
>
> See? This is why *even after I tested it* I wasn't sure I was right!
> The rules are... complicated.

Hence my doubts about this function - it has almost no chance of not
being broken, potentially leading to whatever code was using it having
security holes. Perhaps it should be deprecated or at least attract a
large caveat in the documentation.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to XML?

[Frank Millman]

"Joonas Liik"  wrote in message 
news:cab1gnpqnjdenaa-gzgt0tbcvwjakngd3yroixgyy+mim7fw...@mail.gmail.com...



On 26 August 2016 at 08:22, Frank Millman  wrote:
>
> So this is my conversion routine -
>
> lines = string.split('"')  # split on attributes
> for pos, line in enumerate(lines):
>if pos%2:  # every 2nd line is an attribute
>lines[pos] = line.replace('<', '<').replace('>', '>')
> return '"'.join(lines)
>

or.. you could just escape all & as & before escaping the > and <,
and do the reverse on decode



Thanks, Joonas, but I have not quite grasped that.

Would you mind explaining how it would work?

Just to confirm that we are talking about the same thing -

This is not allowed - ''  [A]


import xml.etree.ElementTree as etree
x = ''
y = etree.fromstring(x)

Traceback (most recent call last):
 File "", line 1, in 
 File 
"C:\Users\User\AppData\Local\Programs\Python\Python35\lib\xml\etree\ElementTree.py", 
line 1320, in XML

   parser.feed(text)
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, 
column 17


You have to escape it like this - '' 
[B]



x = ''
y = etree.fromstring(x)
y.find('fld').get('name')

''




I want to convert the string from [B] to [A] for editing, and then back to 
[B] before saving.


Thanks

Frank


--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 raising an error where Python 2 did not

[d...@forestfield.co.uk]

Thanks for the replies. My example seems to be from the fairly harmless end of 
a wedge of behaviours that are being handled much more sensibly in Python 3.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to XML?

[Joonas Liik]

On 26 August 2016 at 16:10, Frank Millman  wrote:
> "Joonas Liik"  wrote in message
> news:cab1gnpqnjdenaa-gzgt0tbcvwjakngd3yroixgyy+mim7fw...@mail.gmail.com...
>
>> On 26 August 2016 at 08:22, Frank Millman  wrote:
>> >
>> > So this is my conversion routine -
>> >
>> > lines = string.split('"')  # split on attributes
>> > for pos, line in enumerate(lines):
>> >if pos%2:  # every 2nd line is an attribute
>> >lines[pos] = line.replace('<', '<').replace('>', '>')
>> > return '"'.join(lines)
>> >
>>
>> or.. you could just escape all & as & before escaping the > and <,
>> and do the reverse on decode
>>
>
> Thanks, Joonas, but I have not quite grasped that.
>
> Would you mind explaining how it would work?
>
> Just to confirm that we are talking about the same thing -
>
> This is not allowed - ''  [A]
>
 import xml.etree.ElementTree as etree
 x = ''
 y = etree.fromstring(x)
>
> Traceback (most recent call last):
>  File "", line 1, in 
>  File
> "C:\Users\User\AppData\Local\Programs\Python\Python35\lib\xml\etree\ElementTree.py",
> line 1320, in XML
>parser.feed(text)
> xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1,
> column 17
>
> You have to escape it like this - ''
> [B]
>
 x = ''
 y = etree.fromstring(x)
 y.find('fld').get('name')
>
> ''


>
> I want to convert the string from [B] to [A] for editing, and then back to
> [B] before saving.
>
> Thanks
>
> Frank
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

something like.. (untested)

def escape(untrusted_string):
''' Use on the user provided strings to render them inert for storage
  escaping & ensures that the user cant type sth like '>' in
source and have it magically decode as '>'
'''
return untrusted_string.replace("&","&").replace("<",
"<").replace(">", ">")

def unescape(escaped_string):
'''Once the user string is retreived from storage use this
function to restore it to its original form'''
return escaped_string.replace("<","<").replace(">",
">").replace("&", "&")

i should note tho that this example is very ad-hoc, i'm no xml expert
just know a bit about xml entities.
if you decide to go this route there are probably some much better
tested functions out there to escape text for storage in xml
documents.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to XML?

[Joonas Liik]

On 26 August 2016 at 17:58, Joonas Liik  wrote:
> On 26 August 2016 at 16:10, Frank Millman  wrote:
>> "Joonas Liik"  wrote in message
>> news:cab1gnpqnjdenaa-gzgt0tbcvwjakngd3yroixgyy+mim7fw...@mail.gmail.com...
>>
>>> On 26 August 2016 at 08:22, Frank Millman  wrote:
>>> >
>>> > So this is my conversion routine -
>>> >
>>> > lines = string.split('"')  # split on attributes
>>> > for pos, line in enumerate(lines):
>>> >if pos%2:  # every 2nd line is an attribute
>>> >lines[pos] = line.replace('<', '<').replace('>', '>')
>>> > return '"'.join(lines)
>>> >
>>>
>>> or.. you could just escape all & as & before escaping the > and <,
>>> and do the reverse on decode
>>>
>>
>> Thanks, Joonas, but I have not quite grasped that.
>>
>> Would you mind explaining how it would work?
>>
>> Just to confirm that we are talking about the same thing -
>>
>> This is not allowed - ''  [A]
>>
> import xml.etree.ElementTree as etree
> x = ''
> y = etree.fromstring(x)
>>
>> Traceback (most recent call last):
>>  File "", line 1, in 
>>  File
>> "C:\Users\User\AppData\Local\Programs\Python\Python35\lib\xml\etree\ElementTree.py",
>> line 1320, in XML
>>parser.feed(text)
>> xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1,
>> column 17
>>
>> You have to escape it like this - ''
>> [B]
>>
> x = ''
> y = etree.fromstring(x)
> y.find('fld').get('name')
>>
>> ''
>
>
>>
>> I want to convert the string from [B] to [A] for editing, and then back to
>> [B] before saving.
>>
>> Thanks
>>
>> Frank
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
> something like.. (untested)
>
> def escape(untrusted_string):
> ''' Use on the user provided strings to render them inert for storage
>   escaping & ensures that the user cant type sth like '>' in
> source and have it magically decode as '>'
> '''
> return untrusted_string.replace("&","&").replace("<",
> "<").replace(">", ">")
>
> def unescape(escaped_string):
> '''Once the user string is retreived from storage use this
> function to restore it to its original form'''
> return escaped_string.replace("<","<").replace(">",
> ">").replace("&", "&")
>
> i should note tho that this example is very ad-hoc, i'm no xml expert
> just know a bit about xml entities.
> if you decide to go this route there are probably some much better
> tested functions out there to escape text for storage in xml
> documents.

you might want to un-wrap that before testing tho.. no idea why my
messages get mutilated like that :(
(sent using gmail, maybe somebody can comment on that?)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to XML?

[Frank Millman]

"Joonas Liik"  wrote in message 
news:cab1gnptp0gd4s4kx07r1ujrnuxtoij4vf5unye1cfr_y0xv...@mail.gmail.com...



something like.. (untested)


def escape(untrusted_string):
   ''' Use on the user provided strings to render them inert for storage
 escaping & ensures that the user cant type sth like '>' in
source and have it magically decode as '>'
   '''
   return untrusted_string.replace("&","&").replace("<",
"<").replace(">", ">")

def unescape(escaped_string):
   '''Once the user string is retreived from storage use this
function to restore it to its original form'''
   return escaped_string.replace("<","<").replace(">",
">").replace("&", "&")

i should note tho that this example is very ad-hoc, i'm no xml expert just 
know a bit about xml entities.  if you decide to go this route there are 
probably some much better tested functions out there to escape text for 
storage in xml documents.


Thanks very much, Joonas.

I understand now, and it seems to work fine.

As a bonus, I can now include '&' in my attributes in the future if the need 
arises.


Much appreciated.

Frank


--
https://mail.python.org/mailman/listinfo/python-list

Re: [Bulk] Re: Alternatives to XML?

[Roland Koebler]

Hi,

> It is *my* XML, and I know that I only use the offending characters inside
> attributes, and attributes are the only place where double-quote marks are
> allowed.
> 
> So this is my conversion routine -
> 
> lines = string.split('"')  # split on attributes
> for pos, line in enumerate(lines):
>if pos%2:  # every 2nd line is an attribute
>lines[pos] = line.replace('<', '<').replace('>', '>')
> return '"'.join(lines)
OMG!
So, you have a fileformat, which looks like XML, but actually isn't XML,
and will break if used with some "real" XML.

Although I don't like XML, if you want XML, you should follow Chris advice:
On Thu, Aug 25, 2016 at 09:40:03PM +1000, Chris Angelico wrote:
> just make sure it's always valid XML, rather
> than some "XML-like" file structure.

So, please:
- Don't try to write your own (not-quite-)XML-parser.
- Read how XML-files work.
- Read https://docs.python.org/3/library/xml.html
  and https://pypi.python.org/pypi/defusedxml/
- Think what you have done.
- Use a sensible XML-parser/dumper. This should escape most special-
  characters for you (at least: < > & " ').


Roland
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: wxPython Cookbook

[Gonzales, Dean]

Very cool! Thanks for doing this. I can't wait to dig into your cookbook.

Regards,
Dean Gonzales

-Original Message-
From: Python-announce-list 
[mailto:python-announce-list-bounces+dean.gonzales=amd@python.org] On 
Behalf Of Mike Driscoll
Sent: Wednesday, August 24, 2016 12:57 PM
To: python-announce-l...@python.org
Subject: ANN: wxPython Cookbook

Hi,

Several years ago, the readers of my popular Python blog 
 asked me to take some of my articles and 
turn them into a cookbook on wxPython. I have finally decided to do just that. 
I am including over 50 recipes that I am currently editing to make them more 
consistent and updating them to be compatible with the latest versions of 
wxPython. I currently have nearly 300 pages of content!

If you'd like to check out the funding campaign for the book, you can find it 
here: https://www.kickstarter.com/projects/34257246/wxpython-cookbook/

Thanks,
Mike
--
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to XML?

[Marko Rauhamaa]

"Frank Millman" :

> "Joonas Liik"  wrote in message
> news:cab1gnptp0gd4s4kx07r1ujrnuxtoij4vf5unye1cfr_y0xv...@mail.gmail.com...
>> i should note tho that this example is very ad-hoc, i'm no xml expert
>> just know a bit about xml entities.  if you decide to go this route
>> there are probably some much better tested functions out there to
>> escape text for storage in xml documents.
>
> Thanks very much, Joonas.
>
> I understand now, and it seems to work fine.
>
> As a bonus, I can now include '&' in my attributes in the future if the
> need arises.
>
> Much appreciated.

XML attributes are ridiculously complicated. From the standard:

   Before the value of an attribute is passed to the application or
   checked for validity, the XML processor MUST normalize the attribute
   value by applying the algorithm below, or by using some other method
   such that the value passed to the application is the same as that
   produced by the algorithm.

1. All line breaks MUST have been normalized on input to #xA as
   described in 2.11 End-of-Line Handling, so the rest of this
   algorithm operates on text normalized in this way.

2. Begin with a normalized value consisting of the empty string.

3. For each character, entity reference, or character reference in
   the unnormalized attribute value, beginning with the first and
   continuing to the last, do the following:

* For a character reference, append the referenced character to
  the normalized value.

* For an entity reference, recursively apply step 3 of this
  algorithm to the replacement text of the entity.

* For a white space character (#x20, #xD, #xA, #x9), append a
  space character (#x20) to the normalized value.

* For another character, append the character to the normalized
  value.

   If the attribute type is not CDATA, then the XML processor MUST
   further process the normalized attribute value by discarding any
   leading and trailing space (#x20) characters, and by replacing
   sequences of space (#x20) characters by a single space (#x20)
   character.

   Note that if the unnormalized attribute value contains a character
   reference to a white space character other than space (#x20), the
   normalized value contains the referenced character itself (#xD, #xA
   or #x9). This contrasts with the case where the unnormalized value
   contains a white space character (not a reference), which is replaced
   with a space character (#x20) in the normalized value and also
   contrasts with the case where the unnormalized value contains an
   entity reference whose replacement text contains a white space
   character; being recursively processed, the white space character is
   replaced with a space character (#x20) in the normalized value.

   All attributes for which no declaration has been read SHOULD be
   treated by a non-validating processor as if declared CDATA.

   It is an error if an attribute value contains a reference to an
   entity for which no declaration has been read.

   Following are examples of attribute normalization. Given the
   following declarations:

 
 
 

   the attribute specifications in the left column below would be
   normalized to the character sequences of the middle column if the
   attribute a is declared NMTOKENS and to those of the right columns if
   a is declared CDATA.

   =
   Attribute specification:  a=" 
 
 xyz"
   a is NMTOKENS:x y z
   a is CDATA:   #x20 #x20 x y z
   =
   Attribute specification:  a="&d;&d;A&a; &a;B&da;"
   a is NMTOKENS:A #x20 B
   a is CDATA:   #x20 #x20 A #x20 #x20 #x20 B #x20 #x20
   =
   Attribute specification:  a="

A

B
"
   a is NMTOKENS:#xD #xD A #xA #xA B #xD #xA
   a is CDATA:   #xD #xD A #xA #xA B #xD #xA
   =

   Note that the last example is invalid (but well-formed) if a is
   declared to be of type NMTOKENS.

   https://www.w3.org/TR/REC-xml/#AVNormalize>


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to XML?

[Roland Koebler]

Hi,

after reading the mails of this thread, I would recommend one of the
following ways:

1. Use a computer-readable format and some small editor for humans.
   
   The file-format could then be very simple -- I would recommend JSON.
   Or some kind of database (e.g. SQLite).

   For humans, you would have to write a (small/nice) graphical editor,
   where they can build the logic e.g. by clicking on buttons.
   This can also work for non-programmers, since the graphical editor
   can be adapted to the indended users, give help, run wizards etc.

or:

2. Use a human-readable format and a parser for the computer.

   Then, the fileformat should be optimized for human readability.
   I would recommend a restricted subset of Python. This is much more
   readable/writeable for humans than any XML/JSON/YAML.
   And you could even add a graphical editor to further support
   non-programming-users.

   The computer would then need a special parser. But by using
   Python-expressions (only eval, no exec) and a parser for flow
   control (if/else/for/...) and assignments, this is not too much
   work and is good for many applications.

   I've written such a parser incl. some kind of (pseudo-)sandbox [2]
   for my template-engine "pyratemp" [1], and I've also used it for
   small user-created-procedures.

   [1] http://www.simple-is-better.org/template/pyratemp.html
   [2] It's not a real sandbox -- it's secured only by restricting
   the available commands. If you add unsafe commands to the
   pseudo-sandbox (e.g. Pythons "open"), the user can do bad
   things.
   But without manually adding unsafe commands, I don't know
   any way to get out of this pseudo-sandbox.
   And if you really need a sandbox which is more powerful
   than my pseudo-sandbox, you may want to have a look at
   the sandbox of PyPy.


Trying to use a format which is both directly computer-readable
(without a special parser) and well human readable never really
works well in my experience. Then, you usually have to manually
read/write/edit some kind of parse-tree, which is usually much
harder to read/write than code. But if you want to do this, I
recommend LISP ;).

(By the way: If I did understand your mails correctly, your
program would probably break if someone edits the XML-files
manually, since you're using some kind of XML-like-fileformat
with many non-intuitive assumptions.)


Roland


PS: 

On Wed, Aug 24, 2016 at 04:58:54PM +0200, Frank Millman wrote:
> Here is a JSON version -
> 
> {
>  "case": {
>"compare": {
>  "-src": "_param.auto_party_id",
>  "-op": "is_not",
>  "-tgt": "$None",
>  "case": {
>"on_insert": {
>  "auto_gen": { "-args": "_param.auto_party_id" }
>},
>"not_exists": {
>  "literal": { "-value": "" }
>}
>  }
>}
>  }
> }
I think this is not really good. In JSON, you also have lists, and in this
case, it would probably be better to use some lists instead of dicts, e.g.:

[
["if", ["_param.auto_party_id", "is not", "None"],
["if",   ["on_insert"],  ["set", "value", ["call", "auto_gen", 
"_param.auto_party_id"]]],
["elif", ["not_exists"], ["set", "value", "''"]]
]
]

I think this is much more readable than your XML-code and the
auto-converted JSON.

And it's even less ambigious. (How do you distinguish between the
variable _param.auto_party_id and the string "_param.auto_party_id"
in your XML-example?)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 raising an error where Python 2 did not

[Terry Reedy]

On 8/26/2016 4:50 AM, d...@forestfield.co.uk wrote:

In a program I'm converting to Python 3 I'm examining a list of divisor values, 
some of which can be None, to find the first with a value greater than 1.

Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.

None > 1

False


The response 'False' is not a defined part of the language and no 
program should depend on it.  An earlier version of CPython and another 
implementation of Python 1 or 2 might have said 1 or True.  Then your 
'first divisor > 1' would be None. Not good.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

multiprocessing.pool.Pool.map should take more than one iterable

[ycm . jason]

Hello all,

The official doc describes `Pool.map(func, iterable[, chunksize])` as "A 
parallel equivalent of the map() built-in function.". 

Since the function signature of `map` is `map(function, iterable, ...)`, I 
hereby suggest that `Pool.map` should change its function signature to 
`Pool.map(function, iterable, ... [, chunksize])`. This will bring true 
equivalent to these functions.

Tell me what you think. 

Pool.map: 
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.map

map: https://docs.python.org/3/library/functions.html#map

Jason
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script for searching variable strings between two constant strings

[Steve D'Aprano]

On Sat, 27 Aug 2016 08:33 am, ddream.mercha...@gmail.com wrote:

> My log file has several sections starting with  START  and ending
> with  END   . 

Um. Is this relevant? Are you saying that you only wish to search the file
between those lines, and ignore anything outside of them? If the file looks
like:



 --operation(): AutoAuthOSUserSubmit StartOperation


 START 



 END 


 --operation(): AutoAuthOSUserSubmit StartOperation



do you expect to say that nothing is found?


I'm going to assume that you wouldn't have mentioned this if it wasn't
important, so let's start by filtering out everything outside of
===START=== and ===END=== sections. For that, we want a filter that swaps
between "ignore these lines" and "search these lines" depending on whether
you are inside or outside of a START...END section.

We'll use regular expressions for matching.


import re

START = r'''
(?x)(?# verbose mode)
={2,}   (?# two or more equal signs)
\s* (?# any amount of whitespace)
START   (?# the literal word START in uppercase)
\s* (?# more optional whitespace)
={2,}   (?# two or more equal signs)
$   (?# end of the line)
'''

END = r'={2,}\s*END\s*={2,}$'  # Similar to above, without verbose mode.

START = re.compile(START)
END = re.compile(END)

def filter_sections(lines):
outside = True
for line in lines:
line = line.strip()  # ignore leading and trailing whitespace
if outside:
# ignore all lines until we see START
if re.match(START, line):
outside = False
else:
pass  # just ignore the line
else:
# pass on every line until we see END
if re.match(END, line):
outside = True
else:
yield line



Now you need to test that this does what you expect:


with("mylogfile.log") as f:
for line in filter_sections(f):
print(line)


should print *only* the lines between the START and END lines. Once you are
satisfied that this works correctly, move on to the next part: extracting
the relevant information from each line. There are three things you wish to
look for, so you want three regular expressions. I'm not being paid for
this, so here's one, the other two are up to you:

OPERATION = r'''
(?x)(?# verbose mode)
--operation\(\):(?# literal string)
\s* (?# optional whitespace)
(.*)(?# anything at all, in a group)
\s* (?# more optional whitespace)
StartOperation  (?# another literal string)
.*?$(?# ignore everything to the end of the line)
'''

OPERATION = re.compile(OPERATION)

FOO = ...  # match second thing, similar to above
BAR = ...  # match third thing


Now let's extract the data we want:

def extract(lines):
for line in lines:
line = line.strip()
mo = (re.match(OPERATION, line)
  or re.match(FOO, line) 
  or re.match(BAR, line)
  )
if mo:
yield mo.groups(0)


with open('mylogfile.log') as f:
for match in extract(filter_sections(f)):
print(match)




By the way, the above code is untested.






-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 492: isn't the "await" redundant?

[Gregory Ewing]

Marko Rauhamaa wrote:


How about making *every* function *always* and async, unconditionally?
That way *every* function would be an async and every function call
would be an await.


1. Many people regard it as a feature that you can see where
potential suspension points are.

2. Doing this would require massive changes to the core
interpreter and all C extensions. (The original version of
Stackless Python did something similar, and it was judged
far too big a change to incorporate into CPython.)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Does This Scare You?

[eryk sun]

On Fri, Aug 26, 2016 at 11:28 AM, Jon Ribbens  wrote:
> On 2016-08-24, Chris Angelico  wrote:
>> On Thu, Aug 25, 2016 at 7:00 AM, eryk sun  wrote:
>>> I discovered why "Logs/con.txt" isn't working right in Windows 7,
>>> while "Logs/nul.txt" does get redirected correctly to r"\\.\nul".
>>> Prior to Windows 8 the console doesn't use an NT device, so the base
>>> API has a function named BaseIsThisAConsoleName that looks for names
>>> such as r"\\.CON", r"\\.CONIN$", "CON", or r"C:\Temp\con.txt" and
>>> returns either "CONIN$" or "CONOUT$" if there's a match. A match for
>>> just "CON" maps to one or the other of the latter depending on whether
>>> read or write access is desired.
>>
>> See? This is why *even after I tested it* I wasn't sure I was right!
>> The rules are... complicated.
>
> Hence my doubts about this function - it has almost no chance of not
> being broken, potentially leading to whatever code was using it having
> security holes. Perhaps it should be deprecated or at least attract a
> large caveat in the documentation.

I agree that the docs need a warning that the behavior of paths
containing legacy DOS device names is inconsistent between versions of
Windows and that the rules that Windows uses aren't explicitly
documented.

There's another inconsistency if Python is run under ReactOS (and
probably Wine, too). The ReactOS equivalent to BaseIsThisAConsoleName
is IntCheckForConsoleFileName, which doesn't have the (buggy) speed
hack that Windows uses to gate the more expensive call to
RtlIsDosDeviceName_U:

http://preview.tinyurl.com/reactos-console-c-71210

Also note the comment on line 354 that "C:\some_path\CONIN$" should
open the console. This highlights yet another problem with the current
pathlib check: it doesn't reserve the names "CONIN$" and "CONOUT$".

Actually, the comment is technically wrong for versions prior to
Windows 8. The old implementation of RtlIsDosDeviceName_U only checks
for "CON", not "CONIN$" or "CONOUT$". The latter two devices aren't
inherited from DOS. They were added to the Windows API to allow
opening console handles with both read and write access, e.g. to be
able to call WriteConsoleInput and ReadConsoleOutput. "CON" doesn't
allow this. However, the base API does allow opening plain "CONIN$"
and "CONOUT$" without the "\\.\" device namespace prefix, so those
names are at least reserved for the current directory.

The problem is more encompassing for Windows 8+, which has a real
console device. It no longer calls BaseIsThisAConsoleName to redirect
CreateFile to OpenConsoleW. Instead it passes the regular NT paths
r"\??\CON", r"\??\CONIN$", or r"\??\CONOUT$" to the NtCreateFile
system call, which resolves the object manager symbolic links
respectively to r"\Device\ConDrv\Console",
r"\Device\ConDrv\CurrentIn", and r"\Device\ConDrv\CurrentOut". As part
of the redesign, the base API moved the check for "CONIN$" and
"CONOUT$" into the NT runtime library function
RtlpIsDosDeviceName_Ustr. Now "CON", "CONIN$", and "CONOUT$" are
reserved in every directory, just like how it's always worked for NUL,
AUX, PRN, COM1-9, and LPT1-9. For example:

Windows 10

>>> print(os.path.abspath('C:/Temp/conout$  : spam . eggs'))
\\.\conout$

Windows 7

>>> print(os.path.abspath('C:/Temp/conout$  : spam . eggs'))
C:\Temp\conout$  : spam . eggs
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP 492: isn't the "await" redundant?

[Marko Rauhamaa]

Gregory Ewing :

> Marko Rauhamaa wrote:
>> How about making *every* function *always* and async,
>> unconditionally? That way *every* function would be an async and
>> every function call would be an await.
>
> 1. Many people regard it as a feature that you can see where
> potential suspension points are.

Yeah, it's actually crucial since every suspension point will also
require consideration for alternate stimuli like a possible cancellation
or timeout.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Bulk] Re: Alternatives to XML?

[Frank Millman]

"Roland Koebler"  wrote in message news:20160826140213.GA17438@localhost...


Hi,



OMG!
So, you have a fileformat, which looks like XML, but actually isn't XML, 
and will break if used with some "real" XML.


I don't want to pursue this too much further, but I would like to point out 
that my format is genuine XML. I can back that up with two pieces of 
supporting evidence -


1. After going through the unescape > gui > edit > back_to_application > 
escape routine, I validate the result before saving it -


   try:
   return etree.fromstring(value, parser=self.parser)
except (etree.XMLSyntaxError, ValueError) as e:
   raise AibError(head=self.col_defn.short_descr,
   body='Xml error - {}'.format(e.args[0]))

2. This is how I process the result at run-time. I do not try to parse it 
myself. I convert it into an EtreeElement, and step through it. Each 'tag' 
in the XML maps to a function name in the processing module -


   for xml in elem:
   await globals()[xml.tag](caller, xml)

The built-in ElementTree would work for this, but I actually use lxml, 
because I use a little bit of xpath in my processing, which ElementTree does 
not support.


Frank


--
https://mail.python.org/mailman/listinfo/python-list