Re: What's the purpose the hook method showing in a class definition?
Sibylle Koczian於 2019年10月20日星期日 UTC+8上午2時04分54秒寫道: > Am 19.10.2019 um 13:11 schrieb jf...@ms4.hinet.net: > > For the two examples below: > > (1) > class A: > > ... def foo(self): > > ... self.goo() > > ... > class B(A): > > ... def goo(self): > > ... print(1) > > ... > > > > (2) > class A: > > ... def foo(self): > > ... self.goo() > > ... def goo(self): pass > > ... > class B(A): > > ... def goo(self): > > ... print(1) > > ... > > > > Both can get the same result: > b = B() > b.foo() > > 1 > > > > > What's the benefit of having the hook method goo() in class A in example 2? > > > > --Jach > > > None - as long as nobody tries one of those: > > a = A() > a.foo() > > or > > class C(A): > def foo(self): > super().foo() > print(2) > > c = C() > c.foo() Yes, there will be an attribute error if no goo() was defined. What puzzles me is how a parent's method foo() can find its child's method goo(), no matter it was overwrote or not? MRO won't explain this and I can't find document about it also:-( --Jach -- https://mail.python.org/mailman/listinfo/python-list
Re: What's the purpose the hook method showing in a class definition?
Hi Jach, On 20/10/2019 09:34, jf...@ms4.hinet.net wrote: What puzzles me is how a parent's method foo() can find its child's method goo(), no matter it was overwrote or not? MRO won't explain this and I can't find document about it also:-( This is a generalised description - Python may be slightly different. When foo invokes goo the search for goo starts at the class of the object (which is B), not the class of the executing method (i.e not A). It then proceeds to look for goo up the class hierarchy - first in B, then A then Object. If that fails the RTS modifies the call, to look for a magic method, and starts again at B. When the magic method is found in Object, you get the "not found" error. If you implement the magic method in A or B it will be run instead. Regards Ian -- This email has been checked for viruses by AVG. https://www.avg.com -- https://mail.python.org/mailman/listinfo/python-list
Re: What's the purpose the hook method showing in a class definition?
On Sun, Oct 20, 2019 at 9:06 PM Ian Hobson wrote: > > Hi Jach, > > On 20/10/2019 09:34, jf...@ms4.hinet.net wrote: > > What puzzles me is how a parent's method foo() can find its child's method > > goo(), no matter it was overwrote or not? MRO won't explain this and I > > can't find document about it also:-( > > This is a generalised description - Python may be slightly different. > > When foo invokes goo the search for goo starts at the class of the > object (which is B), not the class of the executing method (i.e not A). > It then proceeds to look for goo up the class hierarchy - first in B, > then A then Object. > Yeah, pretty much. In a simple case of linear inheritance, that's how it works in Python. There are complexities with multiple inheritance and different languages will do things differently, but that doesn't affect this example. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What's the purpose the hook method showing in a class definition?
On 10/20/19 4:34 AM, jf...@ms4.hinet.net wrote: > Yes, there will be an attribute error if no goo() was defined. > > What puzzles me is how a parent's method foo() can find its child's method > goo(), no matter it was overwrote or not? MRO won't explain this and I can't > find document about it also:-( > > --Jach The simple answer is that the attribute lookup happens at run time, not compile time (unlike some other languages). Thus attributes/member functions can be added by sub-classes, or even just to that instance at run time and be found. -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Re: keying by identity in dict and set
Hi Chris, Yes, I am aware of the hash of small integers. But I am not keying with small integers here: I am keying with id() values of class instances. Precisely what my example shows is that the dict/set algorithms in fact *never* call __eq__, when the id() of a class instance is returned by __hash__ (in the implementations of Python I have tested). Please try the code yourself. Tell me what I am missing. What "other problems"? Please provide an example! Thanks! On Sat, Oct 19, 2019 at 9:02 PM Chris Angelico wrote: > > On Sun, Oct 20, 2019 at 3:08 AM Steve White wrote: > > It would appear that if __hash__ returns the id, then that id is used > > internally as the key, and since the id is by definition unique, no > > key collision ever occurs -- at least in every Python implementation > > I've tried. It also seems that, for a class instance obj, > > hash( hash( obj ) ) == hash( obj ) > > hash( id( obj ) ) == id( obj ) > > These are very strong and useful properties. Where are they documented? > > There are two rules that come into play here. One is that smallish > integers use their own value as their hash (so hash(1)==1 etc); the > other is that dictionaries actually look for something that matches on > identity OR equality. That's why using identity instead of equality > will appear to work, even though it can cause other problems when you > mismatch them. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: keying by identity in dict and set
Steve White wrote: > Hi Chris, > > Yes, I am aware of the hash of small integers. But I am not keying > with small integers here: I am keying with id() values of class > instances. The id() values /are/ smallish integers though. (I would guess that this is baked into the CPython source, but did not actually check.) > Precisely what my example shows is that the dict/set algorithms in > fact *never* call __eq__, when the id() of a class instance is > returned by __hash__ (in the implementations of Python I have tested). > Please try the code yourself. Tell me what I am missing. I'd state that a bit differently: (0) Key objects in dicts/sets are only compared if they have the same hash value. The comparison works in two steps: (1) Are both objects the same (a is b --> True)? Then assume they are equal. (2) Are they different objects (a is b --> False)? Take the slow path and actually invoke __eq__ With a sufficiently weird equality: >>> class A: ... def __eq__(self, other): return False ... >>> a = A() >>> items = [a] >>> a in items True >>> [v for v in items if v == a] [] As you can see the hash is not involved (not even defined). > What "other problems"? Please provide an example! > > Thanks! > > On Sat, Oct 19, 2019 at 9:02 PM Chris Angelico wrote: >> >> On Sun, Oct 20, 2019 at 3:08 AM Steve White >> wrote: >> > It would appear that if __hash__ returns the id, then that id is used >> > internally as the key, and since the id is by definition unique, no >> > key collision ever occurs -- at least in every Python implementation >> > I've tried. It also seems that, for a class instance obj, >> > hash( hash( obj ) ) == hash( obj ) >> > hash( id( obj ) ) == id( obj ) >> > These are very strong and useful properties. Where are they >> > documented? >> >> There are two rules that come into play here. One is that smallish >> integers use their own value as their hash (so hash(1)==1 etc); the >> other is that dictionaries actually look for something that matches on >> identity OR equality. That's why using identity instead of equality >> will appear to work, even though it can cause other problems when you >> mismatch them. >> >> ChrisA >> -- >> https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
tri.token compared to Enum
On 10/20/2019 04:16 AM, Anders Hovmöller wrote: On 20 Oct 2019, at 12:41, Steve Jorgensen wrote: Anders Hovmöller wrote: We try to do the same thing in various libraries. We've settled on using existing python and end up with syntax like: class MyForm(Form): field = Field() or in your case class Colors(TokenContainer): red = Token() green = Token() blue = Token() (this is using tri.token). This seems like a good approach and is similar to the strategy Enum uses. Site note: When we saw enums landing we hoped we could ditch tri.token but unfortunately enums have a bunch of limitations and restrictions that make them quite unusable for our more general use case :( Out of curiosity, which limitations? -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: keying by identity in dict and set
Hi Peter, Yes you are right. In fact, I shouldn't even have mentioned the hash() function... it came from a line of reasoning about what an implementation might do if very large integers were returned by __hash__(), and some remarks about the value returned by id() applied to small integers. The point is, I don't think __eq__() is ever called in a situation as described in my post, yet the Python documentation states that if instances are to be used as keys, it must not be used to determine if non-identical instances are equivalent. This puts developers like me in a rather bad position. The code seems to work as it is --- but we don't want to put out code that contravenes the documentation. The options for following the documentation in this situation are: either subject users to unfamiliar, custom-made container classes, or give up the semantics of the "==" operator. It seems so unnecessary, given (my understanding of) how the classes actually function. What can be done? I would be willing to consider a technical solution. I would like to see the documentation revised to reflect the actual workings of the existing container classes. A couple of sentences in the documentation of __hash__ could fix the problem. Is that too much to hope for? How would I proceed to get somebody to look at this? Thanks! On Sun, Oct 20, 2019 at 5:16 PM Peter Otten <__pete...@web.de> wrote: > > Steve White wrote: > > > Hi Chris, > > > > Yes, I am aware of the hash of small integers. But I am not keying > > with small integers here: I am keying with id() values of class > > instances. > > The id() values /are/ smallish integers though. > > (I would guess that this is baked into the CPython source, but did not > actually check.) > > > Precisely what my example shows is that the dict/set algorithms in > > fact *never* call __eq__, when the id() of a class instance is > > returned by __hash__ (in the implementations of Python I have tested). > > Please try the code yourself. Tell me what I am missing. > > I'd state that a bit differently: > > (0) Key objects in dicts/sets are only compared if they have the same hash > value. > > The comparison works in two steps: > > (1) Are both objects the same (a is b --> True)? Then assume they are equal. > > (2) Are they different objects (a is b --> False)? Take the slow path and > actually invoke __eq__ > > With a sufficiently weird equality: > > >>> class A: > ... def __eq__(self, other): return False > ... > >>> a = A() > >>> items = [a] > >>> a in items > True > >>> [v for v in items if v == a] > [] > > As you can see the hash is not involved (not even defined). > > > What "other problems"? Please provide an example! > > > > Thanks! > > > > On Sat, Oct 19, 2019 at 9:02 PM Chris Angelico wrote: > >> > >> On Sun, Oct 20, 2019 at 3:08 AM Steve White > >> wrote: > >> > It would appear that if __hash__ returns the id, then that id is used > >> > internally as the key, and since the id is by definition unique, no > >> > key collision ever occurs -- at least in every Python implementation > >> > I've tried. It also seems that, for a class instance obj, > >> > hash( hash( obj ) ) == hash( obj ) > >> > hash( id( obj ) ) == id( obj ) > >> > These are very strong and useful properties. Where are they > >> > documented? > >> > >> There are two rules that come into play here. One is that smallish > >> integers use their own value as their hash (so hash(1)==1 etc); the > >> other is that dictionaries actually look for something that matches on > >> identity OR equality. That's why using identity instead of equality > >> will appear to work, even though it can cause other problems when you > >> mismatch them. > >> > >> ChrisA > >> -- > >> https://mail.python.org/mailman/listinfo/python-list > > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: keying by identity in dict and set
Steve White wrote: > Hi Peter, > > Yes you are right. In fact, I shouldn't even have mentioned the > hash() function... it came from a line of reasoning about what an > implementation might do if very large integers were returned by > __hash__(), and some remarks about the value returned by id() applied > to small integers. > > The point is, I don't think __eq__() is ever called in a situation as > described in my post, yet the Python documentation states that if > instances are to be used as keys, it must not be used to determine if > non-identical instances are equivalent. What else should be used if not __eq__()? Where in the docs did you you see such a statement? The only limitation for a working dict or set is that for its keys or elements (1) a == b implies hash(a) == hash(b) (2) a == a Does your custom class violate one of the above? -- https://mail.python.org/mailman/listinfo/python-list
Re: keying by identity in dict and set
On Sun, Oct 20, 2019 at 7:57 PM Peter Otten <__pete...@web.de> wrote: > > Steve White wrote: > > > > The point is, I don't think __eq__() is ever called in a situation as > > described in my post, yet the Python documentation states that if > > instances are to be used as keys, it must not be used to determine if > > non-identical instances are equivalent. > > What else should be used if not __eq__()? Where in the docs did you you see > such a statement? > Nothing. Nowhere. As I said, it appears that if __hash__ returns values from the id() function, no collision ever occurs, so there is tie to break in a lookup, so in particular there is no need for the heuristic of calling __eq__. And again, the problem here is that the documentation is a little weak on just how these things really interact. We're having to guess, experiment, and even look at C code of the Python implementations. > The only limitation for a working dict or set is that for its keys or > elements > > (1) a == b implies hash(a) == hash(b) > (2) a == a > > Does your custom class violate one of the above? > Yes. It's all in the original post, including code. Give it a shot! Thanks! -- https://mail.python.org/mailman/listinfo/python-list
Re: keying by identity in dict and set
On Mon, Oct 21, 2019 at 4:33 AM Steve White wrote: > The options for following the documentation in this situation are: > either subject users to unfamiliar, custom-made container classes, or > give up the semantics of the "==" operator. > > It seems so unnecessary, given (my understanding of) how the classes > actually function. > > What can be done? I would be willing to consider a technical solution. > > I would like to see the documentation revised to reflect the actual > workings of the existing container classes. A couple of sentences in > the documentation of __hash__ could fix the problem. The definition of __hash__ is still correct. What you're looking at is a very specific situation wherein an object is not equal to itself - that's incredibly rare (in the core language, the ONLY thing that can ever be like this is a NaN). The dictionary's rule is that it finds any object that is either identical to, or equal to, the thing you asked for. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: keying by identity in dict and set
Steve White wrote: > On Sun, Oct 20, 2019 at 7:57 PM Peter Otten <__pete...@web.de> wrote: >> >> Steve White wrote: >> > >> > The point is, I don't think __eq__() is ever called in a situation as >> > described in my post, yet the Python documentation states that if >> > instances are to be used as keys, it must not be used to determine if >> > non-identical instances are equivalent. >> >> What else should be used if not __eq__()? Where in the docs did you you >> see such a statement? >> > Nothing. Nowhere. > > As I said, it appears that if __hash__ returns values from the id() > function, no collision ever occurs, so there is tie to break in a lookup, > so in particular there is no need for the heuristic of calling __eq__. > > And again, the problem here is that the documentation is a little weak > on just how these things really interact. We're having to guess, > experiment, and even look at C code of the Python implementations. > >> The only limitation for a working dict or set is that for its keys or >> elements >> >> (1) a == b implies hash(a) == hash(b) >> (2) a == a >> >> Does your custom class violate one of the above? >> > Yes. It's all in the original post, including code. > Give it a shot! > > Thanks! OK. $ cat tmp.py class A: def __init__(self, hash): self._hash = hash def __eq__(self, other ): raise Exception("__eq__") def __hash__(self): return self._hash def same_hash(x): return hash(A(x)) == x $ python3 -i tmp.py >>> same_hash(2**63) False >>> same_hash(2**63-1) True The limit for smallish integers seems to be 2**63 for 64-bit CPython. As id() in CPython is the object's memory address you need data allocated above and below that limit to produce a hash collision and trigger __eq__. That appears to be rather unlikely. But even if I have this right it looks like you are relying on internals and I lack the fantasy to imagine a compelling use case. -- https://mail.python.org/mailman/listinfo/python-list
Re: Convert a scientific notation to decimal number, and still keeping the data format as float64
doganad...@gmail.com writes: > > In the meanwhile I have checked Scala , and it's more limited then Python. > As an example: > 0.0001 > 1.0E-4: Double > Why do you think this means Scala is more limited than Python? -- Piet van Oostrum WWW: http://piet.vanoostrum.org/ PGP key: [8DAE142BE17999C4] -- https://mail.python.org/mailman/listinfo/python-list
Fwd: keying by identity in dict and set
-- Forwarded message - From: Steve White Date: Sun, Oct 20, 2019 at 11:38 PM Subject: Re: keying by identity in dict and set To: Peter Otten <__pete...@web.de> Hi Peter, Thanks, that does seem to indicate something. (But there was no need to define a class... you're basically saying >>> hash(2**63-1) == 2**63-1 True >>> hash(2**63) == 2**63 False } As was pointed out in a previous posting though, hash() really doesn't come into the question. It was perhaps a mistake of me to bring it up... As to fantasy... I am a little surprised that some people find such a thing surprising. In other environments, I've often refereed to objects directly by their identity... in C and C++ this is often handled simply by using pointers. Anyway... does anybody have a suggestion of how best to bring this up to the powers that be? A little further explanation would have saved me a great deal of time (even if the answer is that for some reason one must never never do such a thing.) Thanks! On Sun, Oct 20, 2019 at 10:15 PM Peter Otten <__pete...@web.de> wrote: > > Steve White wrote: > > > On Sun, Oct 20, 2019 at 7:57 PM Peter Otten <__pete...@web.de> wrote: > >> > >> Steve White wrote: > >> > > >> > The point is, I don't think __eq__() is ever called in a situation as > >> > described in my post, yet the Python documentation states that if > >> > instances are to be used as keys, it must not be used to determine if > >> > non-identical instances are equivalent. > >> > >> What else should be used if not __eq__()? Where in the docs did you you > >> see such a statement? > >> > > Nothing. Nowhere. > > > > As I said, it appears that if __hash__ returns values from the id() > > function, no collision ever occurs, so there is tie to break in a lookup, > > so in particular there is no need for the heuristic of calling __eq__. > > > > And again, the problem here is that the documentation is a little weak > > on just how these things really interact. We're having to guess, > > experiment, and even look at C code of the Python implementations. > > > >> The only limitation for a working dict or set is that for its keys or > >> elements > >> > >> (1) a == b implies hash(a) == hash(b) > >> (2) a == a > >> > >> Does your custom class violate one of the above? > >> > > Yes. It's all in the original post, including code. > > Give it a shot! > > > > Thanks! > > OK. > > $ cat tmp.py > class A: > def __init__(self, hash): > self._hash = hash > def __eq__(self, other ): > raise Exception("__eq__") > def __hash__(self): > return self._hash > > def same_hash(x): > return hash(A(x)) == x > $ python3 -i tmp.py > >>> same_hash(2**63) > False > >>> same_hash(2**63-1) > True > > The limit for smallish integers seems to be 2**63 for 64-bit CPython. As > id() in CPython is the object's memory address you need data allocated above > and below that limit to produce a hash collision and trigger __eq__. That > appears to be rather unlikely. > > But even if I have this right it looks like you are relying on internals and > I lack the fantasy to imagine a compelling use case. > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
[ANN] Austin -- CPython frame stack sampler v1.0.0 is now available
I am delighted to announce the release 1.0.0 of Austin. If you haven't heard of Austin before, it is a frame stack sampler for CPython. It can be used to obtain statistical profiling data out of a running Python application without a single line of instrumentation. This means that you can start profiling a Python application straightaway, even while it's running on a production environment, with minimal impact on performance. The simplest way of using Austin is by piping its output to FlameGraph for a quick and detailed representation of the collected samples. The latest release introduces a memory profiling mode which allows you to profile memory usage. Austin is a pure C application that has no other dependencies other than the C standard library. Its source code is hosted on GitHub at https://github.com/P403n1x87/austin The README contains installation and usage details, as well as some examples of Austin in action. Details on how to contribute to Austin's development can be found at the bottom of the page. All the best, Gabriele What's New == - Added support for multi-process Python applications. - Added support for Python 3.8. Bugfixes - Fixed support for WSL on Windows. -- https://mail.python.org/mailman/listinfo/python-list
Re: Convert a scientific notation to decimal number, and still keeping the data format as float64
my statement may seem unlogical while evaluating and comparing the languages as a whole.. I thought when I give a small number into the programme , the more decimals I can see after the dot as an output, the more human readable it is. when I see a bunch of numbers with 'e' s I know the numbers are small but it is hard for me to compare it to other numbers with 'e'. , specially with the human eye. I dont know much about scala actually. I have just have tried to give 0.0001 and it returned a presentation with an 'e' .whereas python takes 0.0001 and gives 0.0001 . it made me think python is better in that specific subject. However, python though starts to give 'e' number when 5 decimals are given as input. Although there can be systems around which are better in this subject other things I can achieve in python overrides some disadvantages. -- https://mail.python.org/mailman/listinfo/python-list
Re: What's the purpose the hook method showing in a class definition?
Ian Hobson於 2019年10月20日星期日 UTC+8下午6時05分11秒寫道: > Hi Jach, > > On 20/10/2019 09:34, jf...@ms4.hinet.net wrote: > > What puzzles me is how a parent's method foo() can find its child's method > > goo(), no matter it was overwrote or not? MRO won't explain this and I > > can't find document about it also:-( > > This is a generalised description - Python may be slightly different. > > When foo invokes goo the search for goo starts at the class of the > object (which is B), not the class of the executing method (i.e not A). > It then proceeds to look for goo up the class hierarchy - first in B, > then A then Object. > > If that fails the RTS modifies the call, to look for a magic method, and > starts again at B. When the magic method is found in Object, you get the > "not found" error. If you implement the magic method in A or B it will > be run instead. > > Regards > > Ian > > -- > This email has been checked for viruses by AVG. > https://www.avg.com I see. An obj.method will be resolved according to the MRO of the obj's class, no matter where it is located. Thank you. --Jach -- https://mail.python.org/mailman/listinfo/python-list
Re: Convert a scientific notation to decimal number, and still keeping the data format as float64
On Sun, Oct 20, 2019 at 6:06 PM wrote: > > > my statement may seem unlogical while evaluating and comparing the languages > as a whole.. > > I thought when I give a small number into the programme , the more decimals I > can see after the dot as an output, the more human readable it is. > > when I see a bunch of numbers with 'e' s I know the numbers are small but it > is hard for me to compare it to other numbers with 'e'. , specially with the > human eye. > > I dont know much about scala actually. I have just have tried to give 0.0001 > and it returned a presentation with an 'e' .whereas python takes 0.0001 and > gives 0.0001 . it made me think python is better in that specific subject. > > However, python though starts to give 'e' number when 5 decimals are given as > input. Although there can be systems around which are better in this > subject other things I can achieve in python overrides some disadvantages. > -- > https://mail.python.org/mailman/listinfo/python-list The question here has nothing to do with a programming language. As a bystander, this thread is interesting in a strange way. We have a poster who has no programming experience, no background to understand floating point numbers, no concept of the difference between an internal computer usable representation of a number, and a human readable representation. I am sure I am much older than many people here. When I first learned about computers, and programming, and number formats, I also learned about binary numbers, how computers perform arithmetic calculations at a bit level, and lots of things no one really cares about today. Fast forward and we have a person who (I'm guessing) is trying to write a program to complete some schoolwork in a subject far afield for learning about computer programming. Although the problem to be solved seems to be statistical or somehow numeric, the poster doesn't seem to understand so much about the math either, having gone haywire over exponential notation. It is great that computers are so commonplace that problems can be studied and solved with them when the problem solver has so little basic understanding of the tool he is using. But the thread shows the downside to lacking the basics about the tool. I don't mean any negative connotation toward to original poster. The problem he poses is legitimate, and confounding for him and for those who have tried to answer him here since there are these hard disconnects about concepts that are required to understand the question. Its a problem that everyone confronts daily -- No one really knows how anything works under the hood. Think: car, toaster, microwave oven, Facebook algorithms, light bulbs, and on and on -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays -- https://mail.python.org/mailman/listinfo/python-list
In praise of triple-quoted strings
Python's triple-quoted multiline string is a wonderful feature. It vastly simplifies the writing of polyglot scripts (where the same file contains multiple different languages of code in it), hiding any non-Python code inside a triple quoted string. Had need of a Makefile+Python setup today... (Note that if the formatting is messed up, just make sure the "@python3 Makefile" line is indented with exactly one tab.) # Makefile+Python script hide_from_python = """ " all: README.md @python3 Makefile define now_hide_from_make = " """ # At this point, it's all Python. :) import this print("Hello, world") # Postscript to clean everything up. """ " endef end_all_hiding = " """ --- The same trick can be done with quite a few other languages. The non-Python part might be a shell script that locates an interpreter, an Markdown file that explains what to do if you accidentally displayed this file instead of downloading it, a Pike server that runs a supporting server (that one's easy since Pike has a multiline string spelled #" ... ", which looks like a comment to Python), a systemd service file to symlink into your jobs directory... you name it. The only hard part is making the OTHER language ignore the parts it needs to ignore! :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list