Re: How to start gnuradio
Thank you all for your kind explanations. -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 31/07/2018 16:52, Chris Angelico wrote: On Wed, Aug 1, 2018 at 1:28 AM, MRAB wrote: On 2018-07-31 08:40, Robin Becker wrote: A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings . The warning looks wrong to be. In Python 2, u'a' and b'a' would be treated as the same key, but in Python 3 they are distinct and can co-exist. Something for Python's bug tracker, I think! It's a warning specifically requested by the -b option. The two keys in question have the same hash, which means they have to be compared directly; they will compare unequal, but because of the -b flag, the comparison triggers a warning. If that warning is spurious, *don't use the -b option*. ChrisA I looked at the language documentation for 3.7 mappings These represent finite sets of objects indexed by nearly arbitrary values. The only types of values not acceptable as keys are values containing lists or dictionaries or other mutable types that are compared by value rather than by object identity, the reason being that the efficient implementation of dictionaries requires a key’s hash value to remain constant. Numeric types used for keys obey the normal rules for numeric comparison: if two numbers compare equal (e.g., 1 and 1.0) then they can be used interchangeably to index the same dictionary entry. it says explicitly that numeric keys will use numeric comparison, but no mention is made of strings/bytes etc etc and there's an implication that object identity is used rather than comparison. In python 3.x b'a' is not the same as 'a' either the documentation is lacking some reference to strings/bytes or the warning is wrong. Using the excuse that normal comparison is being used seems a bit feeble. It would clearly improve speed if object identity were to be used, but I suppose that's not the case for other reasons. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote:
> On 31/07/2018 16:52, Chris Angelico wrote:
>>
>> On Wed, Aug 1, 2018 at 1:28 AM, MRAB wrote:
>>>
>>> On 2018-07-31 08:40, Robin Becker wrote:
A bitbucket user complains that python 3.6.6 with -Wall -b prints
warnings
>
> .
>>>
>>> The warning looks wrong to be.
>>>
>>> In Python 2, u'a' and b'a' would be treated as the same key, but in
>>> Python 3
>>> they are distinct and can co-exist.
>>>
>>> Something for Python's bug tracker, I think!
>>
>>
>> It's a warning specifically requested by the -b option. The two keys
>> in question have the same hash, which means they have to be compared
>> directly; they will compare unequal, but because of the -b flag, the
>> comparison triggers a warning. If that warning is spurious, *don't use
>> the -b option*.
>>
>> ChrisA
>>
>
> I looked at the language documentation for 3.7 mappings
>
>> These represent finite sets of objects indexed by nearly arbitrary values.
>> The only types of values not acceptable as keys are values containing lists
>> or dictionaries or other mutable types that are compared by value rather
>> than by object identity, the reason being that the efficient implementation
>> of dictionaries requires a key’s hash value to remain constant. Numeric
>> types used for keys obey the normal rules for numeric comparison: if two
>> numbers compare equal (e.g., 1 and 1.0) then they can be used
>> interchangeably to index the same dictionary entry.
>
>
>
>
> it says explicitly that numeric keys will use numeric comparison, but no
> mention is made of strings/bytes etc etc and there's an implication that
> object identity is used rather than comparison. In python 3.x b'a' is not
> the same as 'a' either the documentation is lacking some reference to
> strings/bytes or the warning is wrong. Using the excuse that normal
> comparison is being used seems a bit feeble. It would clearly improve speed
> if object identity were to be used, but I suppose that's not the case for
> other reasons.
>
Technically, the comparison used is:
a is b or a == b
in other words, identity will match, but mainly, equality is used. The
identity check improves performance in many common cases, and also
avoids pathological cases involving float("nan"), but in general
discussion, it's assumed that value rather than identity is the
comparison used.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 09:52, Chris Angelico wrote:
On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote:
On 31/07/2018 16:52, Chris Angelico wrote:
..
it says explicitly that numeric keys will use numeric comparison, but no
.
Technically, the comparison used is:
a is b or a == b
in other words, identity will match, but mainly, equality is used. The
identity check improves performance in many common cases, and also
avoids pathological cases involving float("nan"), but in general
discussion, it's assumed that value rather than identity is the
comparison used.
ChrisA
If the testing were done in that order then no comparison warning would occur
as inspection shows 'a' is not b'a'
> C:\code\hg-repos\reportlab\tmp>\python37\python.exe -b -Wall
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
'a' is b'a'
False
so presumably the testing goes something like
a is b or (a==b if comparable(a,b) else False)
even so I still think errors/warnings created internally by implementers should not be exposed. It's not a big deal. I'm a bit
surprised that we don't have a mapping which uses only identity as that would be faster.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
Robin Becker wrote:
> On 01/08/2018 09:52, Chris Angelico wrote:
>> On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote:
>>> On 31/07/2018 16:52, Chris Angelico wrote:
>>..
>>>
>>> it says explicitly that numeric keys will use numeric comparison, but no
> .
>>>
>>
>> Technically, the comparison used is:
>>
>> a is b or a == b
>>
>> in other words, identity will match, but mainly, equality is used. The
>> identity check improves performance in many common cases, and also
>> avoids pathological cases involving float("nan"), but in general
>> discussion, it's assumed that value rather than identity is the
>> comparison used.
>>
>> ChrisA
>>
>
> If the testing were done in that order then no comparison warning would
> occur as inspection shows 'a' is not b'a'
Nope, that would be the effect of "and", not "or".
>>> "a" is b"b" and "fallback"
False
>>> "a" is b"b" or "fallback"
'fallback'
You seem to be caught in your wrong mental model. I recommend that you let
the matter rest for a day or so, and then look at it with a fresh eye.
> > C:\code\hg-repos\reportlab\tmp>\python37\python.exe -b -Wall
>> Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64
>> bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license"
>> for more information.
> 'a' is b'a'
>> False
>
>
> so presumably the testing goes something like
>
> a is b or (a==b if comparable(a,b) else False)
>
> even so I still think errors/warnings created internally by implementers
> should not be exposed. It's not a big deal. I'm a bit surprised that we
> don't have a mapping which uses only identity as that would be faster.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Wed, Aug 1, 2018 at 7:43 PM, Robin Becker wrote:
> On 01/08/2018 09:52, Chris Angelico wrote:
>>
>> On Wed, Aug 1, 2018 at 6:36 PM, Robin Becker wrote:
>>>
>>> On 31/07/2018 16:52, Chris Angelico wrote:
>>
>> ..
>>>
>>>
>>> it says explicitly that numeric keys will use numeric comparison, but no
>
> .
>>>
>>>
>>
>> Technically, the comparison used is:
>>
>> a is b or a == b
>>
>> in other words, identity will match, but mainly, equality is used. The
>> identity check improves performance in many common cases, and also
>> avoids pathological cases involving float("nan"), but in general
>> discussion, it's assumed that value rather than identity is the
>> comparison used.
>>
>> ChrisA
>>
>
> If the testing were done in that order then no comparison warning would
> occur as inspection shows 'a' is not b'a'
>
> > C:\code\hg-repos\reportlab\tmp>\python37\python.exe -b -Wall
>>
>> Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit
>> (AMD64)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>
> 'a' is b'a'
>>
>> False
>
>
>
> so presumably the testing goes something like
>
> a is b or (a==b if comparable(a,b) else False)
>
> even so I still think errors/warnings created internally by implementers
> should not be exposed. It's not a big deal. I'm a bit surprised that we
> don't have a mapping which uses only identity as that would be faster.
>
I'm not sure how you're interpreting the word 'or' in my explanation,
but it has the same meaning it has in Python code.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
.. Nope, that would be the effect of "and", not "or". "a" is b"b" and "fallback" False "a" is b"b" or "fallback" 'fallback' You seem to be caught in your wrong mental model. I recommend that you let the matter rest for a day or so, and then look at it with a fresh eye. .. my bad; not worrying enough about real problems -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Confused on git commit tree about Lib/datetime.py
On 01/08/18 05:16, Jeffrey Zhang wrote: I found a interesting issue when checking the Lib/datetime.py implementation in python3 This patch is introduced by cf86e368ebd17e10f68306ebad314eea31daaa1e [0]. But if you check the github page[0], or using git tag --contains, you will find v2.7.x includes this commit too. $ git tag --contains cf86e368ebd17e10f68306ebad314eea31daaa1e 3.2 v2.7.10 v2.7.10rc1 v2.7.11 v2.7.11rc1 ... whereas, if you check the v2.7.x code base, nothing is found $ git log v2.7.4 -- Lib/datetime.py I guess it maybe a git tool bug, or the commit tree is messed up. Is there any guys could explain this situation? [0] https://github.com/python/cpython/commit/cf86e368ebd17e10f68306ebad314eea31daaa1e There are some crazy merges in the repository history, that are easy to see with gitk (and not so easy to see with github or the command-line porcelain) In this case, it appears that d26b658f1433a28b611906c078f47bc804a63dd1, in the mercurial days, cherry-picked a patch from the 3.2 branch at f8b9dfd9a1a3d611d43e3c3ef8031fed96c8dd25. I don't know why the git tools don't find this by themselves. I suppose this is a quirk of the hg-git transition and the different ways hg and git have of doing things. -- Thomas -- https://mail.python.org/mailman/listinfo/python-list
Re: Confused on git commit tree about Lib/datetime.py
On Wed, Aug 1, 2018 at 8:40 PM, Thomas Jollans wrote: > On 01/08/18 05:16, Jeffrey Zhang wrote: >> >> I found a interesting issue when checking the Lib/datetime.py >> implementation in python3 >> >> This patch is introduced by cf86e368ebd17e10f68306ebad314eea31daaa1e [0]. >> But if you >> check the github page[0], or using git tag --contains, you will find >> v2.7.x >> includes this commit too. >> >> $ git tag --contains cf86e368ebd17e10f68306ebad314eea31daaa1e >> 3.2 >> v2.7.10 >> v2.7.10rc1 >> v2.7.11 >> v2.7.11rc1 >> ... >> >> whereas, if you check the v2.7.x code base, nothing is found >> >> $ git log v2.7.4 -- Lib/datetime.py >> >> >> I guess it maybe a git tool bug, or the commit tree is messed up. Is there >> any guys could explain this >> situation? >> >> [0] >> >> https://github.com/python/cpython/commit/cf86e368ebd17e10f68306ebad314eea31daaa1e >> > There are some crazy merges in the repository history, that are easy to see > with gitk (and not so easy to see with github or the command-line porcelain) > > In this case, it appears that d26b658f1433a28b611906c078f47bc804a63dd1, in > the mercurial days, cherry-picked a patch from the 3.2 branch at > f8b9dfd9a1a3d611d43e3c3ef8031fed96c8dd25. > > I don't know why the git tools don't find this by themselves. I suppose this > is a quirk of the hg-git transition and the different ways hg and git have > of doing things. I'm not sure what you're expecting them to find. They find that the commit in question is part of the 2.7 branch, so every parent of it is also part of the 2.7 branch. They are correctly reporting it. The commit is a bit weird, but the tools are faithfully reporting the weirdness. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Use a function arg in soup
Hi.
I want to use a function argument as an argument to a bs4 search for attributes.
I had this working not as a function
# noms = soup.findAll('nomination')
# nom_attrs = []
# for attr in soup.nomination.attrs:
# nom_attrs.append(attr)
But as I wanted to keep finding other elements I thought I would turn it into a
generator function.
def attrKey(element):
for attr in soup.element.attrs:
yield attr
results in a Nonetype as soup.element.attrs is passed and the attribute isn't
substituted.
# Attempt 2
def attrKey(element):
for attr in "soup.{}.attrs".format(element):
yield attr
# Attempt 3
def attrKey(element):
search_attr = "soup.{}.attrs".format(element)
for attr in search_attr:
yield attr
so I would call it like
attrKey('nomination')
Any ideas on how the function argument can be used as the search attribute?
Thanks
Sayth
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 8/1/18 4:36 AM, Robin Becker wrote: > On 31/07/2018 16:52, Chris Angelico wrote: >> On Wed, Aug 1, 2018 at 1:28 AM, MRAB wrote: >>> On 2018-07-31 08:40, Robin Becker wrote: A bitbucket user complains that python 3.6.6 with -Wall -b prints warnings > . >>> The warning looks wrong to be. >>> >>> In Python 2, u'a' and b'a' would be treated as the same key, but in >>> Python 3 >>> they are distinct and can co-exist. >>> >>> Something for Python's bug tracker, I think! >> >> It's a warning specifically requested by the -b option. The two keys >> in question have the same hash, which means they have to be compared >> directly; they will compare unequal, but because of the -b flag, the >> comparison triggers a warning. If that warning is spurious, *don't use >> the -b option*. >> >> ChrisA >> > > I looked at the language documentation for 3.7 mappings > >> These represent finite sets of objects indexed by nearly arbitrary >> values. The only types of values not acceptable as keys are values >> containing lists or dictionaries or other mutable types that are >> compared by value rather than by object identity, the reason being >> that the efficient implementation of dictionaries requires a key’s >> hash value to remain constant. Numeric types used for keys obey the >> normal rules for numeric comparison: if two numbers compare equal >> (e.g., 1 and 1.0) then they can be used interchangeably to index the >> same dictionary entry. > > > > it says explicitly that numeric keys will use numeric comparison, but > no mention is made of strings/bytes etc etc and there's an implication > that object identity is used rather than comparison. In python 3.x > b'a' is not the same as 'a' either the documentation is lacking some > reference to strings/bytes or the warning is wrong. Using the excuse > that normal comparison is being used seems a bit feeble. It would > clearly improve speed if object identity were to be used, but I > suppose that's not the case for other reasons. One problem with using identity for strings is that strings that have the same value may not have the same identity. I believe very short strings, like small numbers are cached, so same valued (and typed) short strings will have the same identity, but longer strings are not. It might work for 'a', but might not for 'employee'. -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Wed, Aug 1, 2018 at 9:56 PM, Richard Damon wrote: >> it says explicitly that numeric keys will use numeric comparison, but >> no mention is made of strings/bytes etc etc and there's an implication >> that object identity is used rather than comparison. In python 3.x >> b'a' is not the same as 'a' either the documentation is lacking some >> reference to strings/bytes or the warning is wrong. Using the excuse >> that normal comparison is being used seems a bit feeble. It would >> clearly improve speed if object identity were to be used, but I >> suppose that's not the case for other reasons. > > One problem with using identity for strings is that strings that have > the same value may not have the same identity. I believe very short > strings, like small numbers are cached, so same valued (and typed) short > strings will have the same identity, but longer strings are not. It > might work for 'a', but might not for 'employee'. Strings and numbers MUST be compared by value, not identity. CPython has a cache for strings used in compilation, but that's about it: >>> s1 = "spam" >>> s2, s3 = "sp", "am" >>> s4 = s2 + s3 >>> s1 is s4, s1 == s4 (False, True) >>> n1 = 42 >>> n2 = n1 * 3 >>> n3 = n2 / 3 >>> n1 is n3, n1 == n3 (False, True) >>> n4 = 123456789 >>> n5 = n4 * 3 >>> n6 = n5 // 3 >>> n4 is n6, n4 == n6 (False, True) The middle example is guaranteed (because n1 is an int, n3 is a float), but the other two could in theory be cached by a compliant Python implementation. I don't know of any that do, though. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Use a function arg in soup
Sayth Renshaw wrote:
> Hi.
>
> I want to use a function argument as an argument to a bs4 search for
> attributes.
>
> I had this working not as a function
># noms = soup.findAll('nomination')
> # nom_attrs = []
> # for attr in soup.nomination.attrs:
> # nom_attrs.append(attr)
> But as I wanted to keep finding other elements I thought I would turn it
> into a generator function.
>
>
> def attrKey(element):
> for attr in soup.element.attrs:
> yield attr
>
> results in a Nonetype as soup.element.attrs is passed and the attribute
> isn't substituted.
(1) soup.nomination will only give the first , so in the general
case soup.find_all("nomination") is the better approach.
(2) attrs is a dict, so iterating over it will lose the values. Are you sure
you want that?
If you use find_all() that already takes a string, so
>>> from bs4 import BeautifulSoup as BS
>>> soup = BS("FOOBARFOO2")
>>> def gen_attrs(soup, element):
... for e in soup.find_all(element): yield e.attrs
...
>>> list(gen_attrs(soup, "foo"))
[{'b': '2', 'a': '1'}, {'d': '42'}]
To access an attribute programatically there's the getattr() builtin;
foo.bar is equivalent to getattr(foo, "bar"), so
>>> def get_attrs(soup, element):
... return getattr(soup, element).attrs
...
>>> get_attrs(soup, "foo")
{'b': '2', 'a': '1'}
>
> # Attempt 2
> def attrKey(element):
> for attr in "soup.{}.attrs".format(element):
> yield attr
>
> # Attempt 3
> def attrKey(element):
> search_attr = "soup.{}.attrs".format(element)
> for attr in search_attr:
> yield attr
>
>
> so I would call it like
> attrKey('nomination')
>
> Any ideas on how the function argument can be used as the search
> attribute?
>
> Thanks
>
> Sayth
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
messing with bytes I discover that this doesn't warn with python -b
if __name__=='__main__':
class nbytes(bytes):
def __eq__(self,other):
return bytes.__eq__(self,other) if isinstance(other,bytes) else
False
def __hash__(self):
return bytes.__hash__(self)
d={'a':1}
d[nbytes(b'a')] = d['a']
print(d)
d={nbytes(b'a'):1}
d['a'] = d[b'a']
print(d)
C:\code\hg-repos\reportlab\tmp>\python37\python -b tb1.py
{'a': 1, b'a': 1}
{b'a': 1, 'a': 1}
I expected one of the assignments to warn.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Wed, Aug 1, 2018 at 11:25 PM, Robin Becker wrote:
> messing with bytes I discover that this doesn't warn with python -b
>
>
> if __name__=='__main__':
> class nbytes(bytes):
> def __eq__(self,other):
> return bytes.__eq__(self,other) if isinstance(other,bytes) else
> False
> def __hash__(self):
> return bytes.__hash__(self)
> d={'a':1}
> d[nbytes(b'a')] = d['a']
> print(d)
> d={nbytes(b'a'):1}
> d['a'] = d[b'a']
> print(d)
>
>
>
>> C:\code\hg-repos\reportlab\tmp>\python37\python -b tb1.py
>> {'a': 1, b'a': 1}
>> {b'a': 1, 'a': 1}
>
>
> I expected one of the assignments to warn.
It's a warning designed to help people port code from Py2 to Py3. It's
not meant to catch every possible comparison. Unless you are actually
porting Py2 code and are worried that you'll be accidentally comparing
bytes and text, just *don't use the -b switch* and there will be no
problems.
I don't understand what the issue is here.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On 01/08/2018 14:38, Chris Angelico wrote: t's a warning designed to help people port code from Py2 to Py3. It's not meant to catch every possible comparison. Unless you are actually porting Py2 code and are worried that you'll be accidentally comparing bytes and text, just*don't use the -b switch* and there will be no problems. I don't understand what the issue is here. I don't either, I have never used the -b flag until the issue was raised on bitbucket. If someone is testing a program with reportlab and uses that flag then they get a lot of warnings from this dictionary assignment. Probably the code needs tightening so that we insist on using native strings everywhere; that's quite hard for py2/3 compatible code. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Wed, 1 Aug 2018 at 16:10, Robin Becker wrote: > > On 01/08/2018 14:38, Chris Angelico wrote: > > t's a warning designed to help people port code from Py2 to Py3. It's > > not meant to catch every possible comparison. Unless you are actually > > porting Py2 code and are worried that you'll be accidentally comparing > > bytes and text, just*don't use the -b switch* and there will be no > > problems. > > > > I don't understand what the issue is here. > > I don't either, I have never used the -b flag until the issue was raised on > bitbucket. If someone is testing a program with > reportlab and uses that flag then they get a lot of warnings from this > dictionary assignment. Probably the code needs tightening > so that we insist on using native strings everywhere; that's quite hard for > py2/3 compatible code. They should probably use the warnings module to disable the warning in library code that they don't control, in that case. If they've reported to you that your code produces warnings under -b, your response can quite reasonably be "thanks for the information, we've reviewed our bytes/string handling and can confirm that it's safe, so there's no fixes needed in reportlab". Paul -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Thu, Aug 2, 2018 at 1:22 AM, Paul Moore wrote: > On Wed, 1 Aug 2018 at 16:10, Robin Becker wrote: >> >> On 01/08/2018 14:38, Chris Angelico wrote: >> > t's a warning designed to help people port code from Py2 to Py3. It's >> > not meant to catch every possible comparison. Unless you are actually >> > porting Py2 code and are worried that you'll be accidentally comparing >> > bytes and text, just*don't use the -b switch* and there will be no >> > problems. >> > >> > I don't understand what the issue is here. >> >> I don't either, I have never used the -b flag until the issue was raised on >> bitbucket. If someone is testing a program with >> reportlab and uses that flag then they get a lot of warnings from this >> dictionary assignment. Probably the code needs tightening >> so that we insist on using native strings everywhere; that's quite hard for >> py2/3 compatible code. > > They should probably use the warnings module to disable the warning in > library code that they don't control, in that case. > > If they've reported to you that your code produces warnings under -b, > your response can quite reasonably be "thanks for the information, > we've reviewed our bytes/string handling and can confirm that it's > safe, so there's no fixes needed in reportlab". Yep, agreed. And I suspect that there may be a bit of non-thinking-C-mentality creeping in: "if I can turn on warnings, I should, and any warning is a problem". That simply isn't the case in Python. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Confused on git commit tree about Lib/datetime.py
On 01/08/18 12:49, Chris Angelico wrote: On Wed, Aug 1, 2018 at 8:40 PM, Thomas Jollans wrote: On 01/08/18 05:16, Jeffrey Zhang wrote: I found a interesting issue when checking the Lib/datetime.py implementation in python3 This patch is introduced by cf86e368ebd17e10f68306ebad314eea31daaa1e [0]. But if you check the github page[0], or using git tag --contains, you will find v2.7.x includes this commit too. $ git tag --contains cf86e368ebd17e10f68306ebad314eea31daaa1e 3.2 v2.7.10 v2.7.10rc1 v2.7.11 v2.7.11rc1 ... whereas, if you check the v2.7.x code base, nothing is found $ git log v2.7.4 -- Lib/datetime.py I guess it maybe a git tool bug, or the commit tree is messed up. Is there any guys could explain this situation? [0] https://github.com/python/cpython/commit/cf86e368ebd17e10f68306ebad314eea31daaa1e There are some crazy merges in the repository history, that are easy to see with gitk (and not so easy to see with github or the command-line porcelain) In this case, it appears that d26b658f1433a28b611906c078f47bc804a63dd1, in the mercurial days, cherry-picked a patch from the 3.2 branch at f8b9dfd9a1a3d611d43e3c3ef8031fed96c8dd25. I don't know why the git tools don't find this by themselves. I suppose this is a quirk of the hg-git transition and the different ways hg and git have of doing things. I'm not sure what you're expecting them to find. They find that the commit in question is part of the 2.7 branch, so every parent of it is also part of the 2.7 branch. They are correctly reporting it. The commit is a bit weird, but the tools are faithfully reporting the weirdness. ChrisA That's not the big I'm confused about. I (like Jeffrey) would have expected git log v2.7.4 -- Lib/datetime.py to show something. But right now I'm not sure that ever shows anything for deleted files, so there's that. Thomas -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
Paul Moore wrote: > On Wed, 1 Aug 2018 at 16:10, Robin Becker wrote: >> >> On 01/08/2018 14:38, Chris Angelico wrote: >> > t's a warning designed to help people port code from Py2 to Py3. It's >> > not meant to catch every possible comparison. Unless you are actually >> > porting Py2 code and are worried that you'll be accidentally comparing >> > bytes and text, just*don't use the -b switch* and there will be no >> > problems. >> > >> > I don't understand what the issue is here. >> >> I don't either, I have never used the -b flag until the issue was raised >> on bitbucket. If someone is testing a program with reportlab and uses >> that flag then they get a lot of warnings from this dictionary >> assignment. Probably the code needs tightening so that we insist on using >> native strings everywhere; that's quite hard for py2/3 compatible code. > > They should probably use the warnings module to disable the warning in > library code that they don't control, in that case. > > If they've reported to you that your code produces warnings under -b, > your response can quite reasonably be "thanks for the information, > we've reviewed our bytes/string handling and can confirm that it's > safe, so there's no fixes needed in reportlab". > > Paul I've looked into the actual code which has # paraparser.py f = isPy3 and asBytes or asUnicode K = list(known_entities.keys()) for k in K: known_entities[f(k)] = known_entities[k] It looks like known_entities starts out with the default string type, i. e. unicode in py3 and bytes in py2. While in py2 the code has no effect in py3 it adds the corresponding keys of type bytes. However, known_entities is then used in HTMLParser.handle_entity_ref(self, name) which passes the name as unicode in py3. I didn't try, but I would suspect that the module keeps working as expected when you remove the lines quoted above. If I'm correct running the program with the -b flag has at least helped in cleaning up the code in this case. In other cases it might detect sources of bugs, so IMHO it's better to have a close look at and possibly rewrite code that triggers the warning rather than to disable it. -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Wed, 01 Aug 2018 16:22:16 +0100, Paul Moore wrote:
> On Wed, 1 Aug 2018 at 16:10, Robin Becker wrote:
>>
>> On 01/08/2018 14:38, Chris Angelico wrote:
>> > t's a warning designed to help people port code from Py2 to Py3. It's
>> > not meant to catch every possible comparison. Unless you are actually
>> > porting Py2 code and are worried that you'll be accidentally
>> > comparing bytes and text, just*don't use the -b switch* and there
>> > will be no problems.
>> >
>> > I don't understand what the issue is here.
>>
>> I don't either, I have never used the -b flag until the issue was
>> raised on bitbucket. If someone is testing a program with reportlab and
>> uses that flag then they get a lot of warnings from this dictionary
>> assignment. Probably the code needs tightening so that we insist on
>> using native strings everywhere; that's quite hard for py2/3 compatible
>> code.
>
> They should probably use the warnings module to disable the warning in
> library code that they don't control, in that case.
>
> If they've reported to you that your code produces warnings under -b,
> your response can quite reasonably be "thanks for the information, we've
> reviewed our bytes/string handling and can confirm that it's safe, so
> there's no fixes needed in reportlab".
I'm sorry, I don't understand this reasoning. (Perhaps I have missed
something.) Robin says his code runs under both Python2 and Python3. He's
getting a warning that the behaviour has changed between the two, and
there's a dubious comparison being made between bytes and strings.
Consequently, there's a very real chance that he has a dicts which have
one key in Python 2 but two in Python 3:
- in Python 2, b'x' and u'x' are the same key;
- in Python 3, b'x' and u'x' are different keys;
# Python 2
py> {u'x': 1, b'x': 2}
{u'x': 2}
#Python 3
py> {u'x': 1, b'x': 2}
{b'x': 2, 'x': 1}
This means that Robin very likely has subtly or not-so-subtly different
behaviour his software depending on which version of Python it runs
under. If not an outright bug that silently does the wrong thing.
Even if Robin has audited the entire code base and can confidently say
today that despite the warning, no such bug has manifested, he cannot
possibly be sure that it won't manifest tomorrow. (Not unless the
software is frozen and will never be modified.)
In another post, Chris says:
I suspect that there may be a bit of non-thinking-C-mentality
creeping in: "if I can turn on warnings, I should, and any
warning is a problem". That simply isn't the case in Python.
I strongly disagree. Unless Chris' intention is to say bugs don't matter
if they're written in Python, I don't know why one would say that
warnings aren't a problem.
Every warning is one of three cases:
- it reveals an existing actual problem;
- it reveals a potential problem which might somebody become
an actual problem;
- or it is a false positive which (if unfixed) distracts
attention and encourages a blasé attitude which could
easily lead to problems in the future.
Warnings are a code smell. Avoiding warnings is a way of "working clean":
https://blog.codinghorror.com/programmers-and-chefs/
Ignoring warnings because they haven't *yet* manifested as a bug, or
worse, because you *assume* that they haven't manifested as a bug, is
about as sensible as ignoring the oil warning light on your car because
the engine hasn't yet seized up. Regardless of which language the
software is written in.
--
Steven D'Aprano
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." -- Jon Ronson
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Wed, 1 Aug 2018 at 18:43, Steven D'Aprano wrote: > > On Wed, 01 Aug 2018 16:22:16 +0100, Paul Moore wrote: > > > If they've reported to you that your code produces warnings under -b, > > your response can quite reasonably be "thanks for the information, we've > > reviewed our bytes/string handling and can confirm that it's safe, so > > there's no fixes needed in reportlab". > > I'm sorry, I don't understand this reasoning. (Perhaps I have missed > something.) Robin says his code runs under both Python2 and Python3. He's > getting a warning that the behaviour has changed between the two, and > there's a dubious comparison being made between bytes and strings. > Consequently, there's a very real chance that he has a dicts which have > one key in Python 2 but two in Python 3: Rightly or wrongly, I'm trusting Robin's assertion that he doesn't believe there's a problem with his code. After all, the change in behaviour between Python 2 and 3 has been explained a couple of times in this thread, so I'm assuming Robin understands it, and is not simply asserting unverified that he thinks his code is OK. Certainly, if Robin hasn't checked that the warning isn't flagging an actual issue with his code, then he should do so. Is that not obvious? If it's not, then my apologies for assuming it was. My point was that it's a *warning*, and as such it's perfectly possible for a warning to *not* need addressing (other than to suppress or ignore it once you're happy that doing so is the right approach). Paul -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
On Thu, Aug 2, 2018 at 3:37 AM, Steven D'Aprano wrote: > In another post, Chris says: > > I suspect that there may be a bit of non-thinking-C-mentality > creeping in: "if I can turn on warnings, I should, and any > warning is a problem". That simply isn't the case in Python. > > I strongly disagree. Unless Chris' intention is to say bugs don't matter > if they're written in Python, I don't know why one would say that > warnings aren't a problem. > > Every warning is one of three cases: > > - it reveals an existing actual problem; > > - it reveals a potential problem which might somebody become > an actual problem; > > - or it is a false positive which (if unfixed) distracts > attention and encourages a blasé attitude which could > easily lead to problems in the future. Right, exactly. And in any code that does not and cannot run on Python 2, the warning about bytes and text comparing unequal is nothing more than a false positive. It's like having a warning saying "In previous versions of Python, this arithmetic operation might have overflowed". In ALL current versions of Python, that is irrelevant noise. Turning on the warning is therefore not a good thing; the non-thinking policy of "turn on all warnings" is not the best default. If a warning is ALWAYS a false positive, it is better to silence it than to have noise that makes *other* warnings less visible. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Are dicts supposed to raise comparison errors
01.08.18 21:03, Chris Angelico пише: And in any code that does not and cannot run on Python 2, the warning about bytes and text comparing unequal is nothing more than a false positive. Not always. If your code supported Python 2 in the past, or third-party dependencies supports or supported Python 2, this warning can expose a real bug. Even if all your and third-party code always was Python 3 only, the standard library can contain such kind of bugs. Several years after the EOL of Python 2.7 and moving all living code to Python 3 we can ignore bytes warnings as always false positive. -- https://mail.python.org/mailman/listinfo/python-list
Dealing with errors in interactive subprocess running python interpreter that freeze the process
I can run python3 interactively in a subprocess w/ Popen but
if I sent it text, that throws an exception, the process freezes
instead of just printing the exception like the normal interpreter..
why? how fix? Here is my code below.
(I suspect when there is an exception, there is NO output to stdin so that
the problem is the line below that tries to read from stdin never finishes.
Maybe I need a different readline that can "survive" when there is no output
and won't block?)
import subprocess
interpreter = subprocess.Popen(['python3', '-i'],
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
while True:
exp = input(">>> ").encode() + b"\n"
interpreter.stdin.write(exp)
interpreter.stdin.flush()
print(interpreter.stdout.readline().strip())
interpreter.stdin.close()
interpreter.terminate()
--
https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
On Thu, Aug 2, 2018 at 6:11 AM, wrote:
> I can run python3 interactively in a subprocess w/ Popen but
> if I sent it text, that throws an exception, the process freezes
> instead of just printing the exception like the normal interpreter..
> why? how fix? Here is my code below.
>
> (I suspect when there is an exception, there is NO output to stdin so that
> the problem is the line below that tries to read from stdin never finishes.
> Maybe I need a different readline that can "survive" when there is no output
> and won't block?)
>
>
>
> import subprocess
>
> interpreter = subprocess.Popen(['python3', '-i'],
>stdin = subprocess.PIPE,
>stdout = subprocess.PIPE,
>stderr = subprocess.PIPE)
>
> while True:
> exp = input(">>> ").encode() + b"\n"
> interpreter.stdin.write(exp)
> interpreter.stdin.flush()
> print(interpreter.stdout.readline().strip())
Normally, I would treat the input and output pipes separately. One
easy (if potentially inefficient) way to do this is to create a thread
for each of the pipes coming back - stdin and stderr, in this case -
and have each one read from one pipe and display it. Then your main
thread can ignore stdout and just push info into the stdin pipe.
> interpreter.stdin.close()
> interpreter.terminate()
Side point: I would recommend joining, rather than forcibly
terminating, the subprocess. Close stdin, then wait for it to finish;
if you're paranoid, wait a limited amount of time and THEN terminate
it, but otherwise, just join() the process.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
RE: Dealing with errors in interactive subprocess running python interpreter that freeze the process
A couple notes:
-I think the Python interpreter actually sends its output to stderr, so to
capture it you'd probably want it to go to the same place as stdout, so use
stderr = subprocess.STDOUT
-You're only reading 1 line out output for each thing, so if 1 command creates
multiple lines of output then you won't be showing them all.
-You're never actually checking if the called process is still alive or not. It
should normally be something like:
...
interpreter = subprocess.Popen(...)
...
interpreter.poll()
while interpreter.returncode is not None:
...
interpreter.poll()
cleanup stuff
-To the actual question it looks like it has the stdout stream in blocking mode
somehow. So when you're reading from stdout and there's nothing there it's
blocking and waiting for there to be something, which will never happen.
Flipping through the documentation for subprocess, and io (interpreter.stdout
is of ) io mentions blocking vs non blocking a lot,
but it's not mentioned in subprocess. And I don't see in either how to tell if
a stream is in blocking mode or not, or how or if it's possible to change that.
So I don't know what to suggest for that, sorry.
-Original Message-
From: Python-list
[mailto:python-list-bounces+david.raymond=tomtom@python.org] On Behalf Of
cseber...@gmail.com
Sent: Wednesday, August 01, 2018 4:11 PM
To: python-list@python.org
Subject: Dealing with errors in interactive subprocess running python
interpreter that freeze the process
I can run python3 interactively in a subprocess w/ Popen but
if I sent it text, that throws an exception, the process freezes
instead of just printing the exception like the normal interpreter..
why? how fix? Here is my code below.
(I suspect when there is an exception, there is NO output to stdin so that
the problem is the line below that tries to read from stdin never finishes.
Maybe I need a different readline that can "survive" when there is no output
and won't block?)
import subprocess
interpreter = subprocess.Popen(['python3', '-i'],
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
while True:
exp = input(">>> ").encode() + b"\n"
interpreter.stdin.write(exp)
interpreter.stdin.flush()
print(interpreter.stdout.readline().strip())
interpreter.stdin.close()
interpreter.terminate()
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Use a function arg in soup
Thanks Peter ### (2) attrs is a dict, so iterating over it will lose the values. Are you sure you want that? ### Yes initially I want just the keys as a list so I can choose to filter them out to only the ones I want. # getAttr Thanks very much will get my function up and working. Cheers Sayth -- https://mail.python.org/mailman/listinfo/python-list
Re: Dealing with errors in interactive subprocess running python interpreter that freeze the process
On 8/1/2018 4:11 PM, cseber...@gmail.com wrote:
I can run python3 interactively in a subprocess w/ Popen but
if I sent it text, that throws an exception, the process freezes
instead of just printing the exception like the normal interpreter..
why? how fix? Here is my code below.
(I suspect when there is an exception, there is NO output to stdin so that
the problem is the line below that tries to read from stdin never finishes.
Maybe I need a different readline that can "survive" when there is no output
and won't block?)
import subprocess
interpreter = subprocess.Popen(['python3', '-i'],
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
while True:
exp = input(">>> ").encode() + b"\n"
interpreter.stdin.write(exp)
interpreter.stdin.flush()
print(interpreter.stdout.readline().strip())
subprocess is not meant for interaction through the pipes. That is why,
I have been told, IDLE uses a socket for interaction. Multiprocess is
apparently better suited for interaction without resorting to a socket.
interpreter.stdin.close()
interpreter.terminate()
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
