Re: Why no list as dict key?
This does raise an issue, Chris, if you use the method of making a tuple companion for a list at a specific time just for use as a dictionary key, then later change the list, you can end up with various situations. Obviously the changed list can not only not access the stored item, but if converted again to a tuple, may address a different item. I can see many scenarios with abandoned dictionary items that are never deleted and can only be reached by examining all items in the dictionary. So mutability is only one concern. If you actually mutate your data, ... I am thinking as an example about a program I wrote ages ago that deals with equations in symbolic form and maintains a collection of forms of the equation it is trying to take a derivative or integral of by applying an assortment of typographic rules. I mean commutative lawand others. You do not want to keep adding the same item into the data structure (such as a queue) repeatedly. So something like a dictionary (or set) can be a good way to store unique items. But the items are some complex lists so the above discussion qualifies. Of course the tuple conversion for a nested structure would need to have made a deep copy. The question in the above is how to make sure that taking a next attempt off the queue deals with the dictionary of tried items. In this case, unless you use it to find a solution, keeping it in the dictionary to avoid repeating, makes sense. And another thought is that mapping a list to a tuple has another possible drawback. What if I have both a list and tuple with the same structure which I want as keys? I can imagine then converting the list in some imaginative ways. For example, embed the list in another list whose first item is "from-tuple" or something. Life is complicated. Then you die. -Original Message- From: Chris Angelico To: python-list@python.org Sent: Wed, Apr 20, 2022 3:49 pm Subject: Re: Why no list as dict key? On Thu, 21 Apr 2022 at 05:30, Sam Ezeh wrote: > > Repeating the above points, here is an example of what would happen if > you tried. Dictionaries require their keys to be immutable as > under-the-hood they use hash tables and they'd fail when the > underlying values are allowed to change. > > ``` > >>> class HashableList(list): > ... def __hash__(self): > ... return functools.reduce(operator.xor, [key * value for > key, value in enumerate(self)], 5) Quickie: I'd be inclined to define hash on top of a tuple's hash, rather than try to design my own and hope that it's suitable. "return hash(tuple(self))" is a good demonstration of the parallel. Otherwise, good demonstration of the problem. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
Python does have a concept of "truthy" that includes meaning for not just the standard Booleans but for 0 and non-zero and the empty string and many more odd things such as an object that defines __bool__ (). But saying it returns a Boolean True/False valuesounds direct and simple and informative enough if that is True. What bothers me is the assumption that anyone knows not so muchjust group theory but what the argument to the function looks like as a Python object of some kind. Does the function accept only some permutation object managed by a specific module? Will it accept some alternate representation such as a list structure or other iterator? Obviously deeper details would normally be in a manual page or other documentation but as "permutations" are likely not to be what most people think about before breakfast, or even after, odd as that may seem, ... And, yes, I know what it means and some users will too. But as all permutations must be even or odd, errors thrown might be based on whether the data structure has valid contents or is so complex that it uses up all system resources, I would think. So the docstring could be fairly short and something like: Given a permutation in Returns the Boolean value True if it is graded as or False if or an exception if the argument is not valid. As noted by others, Many things can be returned including multiple values where perhaps the second one tells if there was an error but thatthe user can ignore or not even catch. -Original Message- From: Chris Angelico To: python-list@python.org Sent: Fri, Apr 22, 2022 6:33 pm Subject: Re: Style for docstring On Sat, 23 Apr 2022 at 08:24, <2qdxy4rzwzuui...@potatochowder.com> wrote: > > On 2022-04-22 at 15:35:15 -0500, > "Michael F. Stemper" wrote: > > > On 22/04/2022 14.59, Chris Angelico wrote: > > > On Sat, 23 Apr 2022 at 05:56, Michael F. Stemper > > > wrote: > > > > > > > > I'm writing a function that is nearly self-documenting by its name, > > > > but still want to give it a docstring. Which of these would be > > > > best from a stylistic point of view: > > > > > > > > > > > > Tells caller whether or not a permutation is even. > > > > > > > > Determines if a permutation is even. (Alternative is that it's odd.) > > > > > > > > Returns True if permutation is even, False if it is odd. > > > > > > > > > > I'd go with the third one, but "Return" rather than "Returns". Or > > > possibly "Test whether a permutation is even". > > > > "So let it be written. So let it be done." > > "Test whether a permutation is even," while technically factual, leaves > the reader to wonder what form the result takes, and what happens to > that result. While it's definitely possible to have other results and other ways to deliver them, the return of a boolean would be the most obvious default. > Do you want callers of the function also to assume that True means that > the permutation is even? There are other reasonable strategies, such as > an enumerated type (whose items are Even, Odd, and FileNotFound), or > throwing an exception if the permutation is odd. I'm assuming that the function is called something like "is_even()" and that it either is a method on a permutation object, or its parameters make it very clear what the permutation is. If it returns an enumeration, I would say that in the docstring. If the docstring doesn't say, I would assume it returns True or False. > I prefer the "return" (rather than "returns") version of the third > option. Assuming that the programmers are familiar with the domain, the > other two leave out important information. Core Python methods and functions seem to prefer either "Return ..." or "Verb the thing" where the result is implicit (eg str.zfill.__doc__ which says "Pad a numeric string..."). Both are used extensively. Neither form leaves out anything that wouldn't be the obvious default. We don't need to say "Figures out algorithmically whether the permutation is even. If it is, will return True; if it isn't, will return False; if something goes wrong, will raise an exception". This is Python; we know that if something goes wrong, an exception is raised. (Though it can help to say WHICH exception will be raised under WHAT circumstances). Some things are obvious. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
We know some people using "professional" language make things shorteror talk from a point of view different than others and often in otherwise incomprehensible jargon. If a programmer is taking about the algorithm that a function implements, then, yes, they may write "scan" and "return". But if they realize the darn documentation is for PEOPLE asking how to use the darn thing, and want to write in more informal and understandable English, I think it makes more sense to say what the function does as in "scans" and importantly what it "returns" to the user as a result. So if you are taking a programming course and the instructor or textbook is giving imperitave commands, they may well tell youto scan something then calculate something and use a return statement a certain way. I can read many ways and am not particularly bothered by either style but when documenting what is, I prefer proper English (or any otherlanguage) in communicating what it does for them. As with many such things, if you work for a company or with groups of others, it is wise to find out what is expected and do the same as much as reasonable. -Original Message- From: MRAB To: python-list@python.org Sent: Fri, Apr 22, 2022 8:57 pm Subject: Re: Style for docstring On 2022-04-23 00:25, Rob Cliffe via Python-list wrote: > I don't use docstrings much; instead I put a line or two of comments > after the `def ` line. > But my practice in such situations is as per the OP's 3rd suggestion, e.g. > # Returns True if . > I'm curious as to why so many people prefer "Return" to "Returns". > Checking out help() on a few functions in the stdlib, they all used > "Return" or a grammatical equivalent, so this does seem to be a Python > cultural thing. But why? To me, "Returns" begins a description as to > what the function does, whereas "Return" is an imperative. But who is > it addresed to? Is a function considered to be a sentient entity that > can respond to a command? Is it an invocation to the lines of code > following the docstring: "Do this!" Might not the programmer mistakenly > think (if only for a moment) that the imperative is addressed to him? Maybe it's because the function name is often also an imperative, e.g.: >>> import re >>> help(re.search) Help on function search in module re: search(pattern, string, flags=0) Scan through string looking for a match to the pattern, returning a Match object, or None if no match was found. Note "Scan", not "scans". I was going to use 'print' as the example: >>> help(print) Help on built-in function print in module builtins: print(...) print(value, ..., sep=' ', end='\n', file=sys.stdout, flush=False) Prints the values to a stream, or to sys.stdout by default. Optional keyword arguments: file: a file-like object (stream); defaults to the current sys.stdout. sep: string inserted between values, default a space. end: string appended after the last value, default a newline. flush: whether to forcibly flush the stream. but it says "Prints", not "Print"... -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
Given what you added, Michael, your function is part of a larger collection of functions and being compatible with the others is a valid consideration. Whatever you decide, would ideally be done consistently with all or most of them. And, of course, it others in the collection also can handle multiple ways to specify a permutation, it may be simpler to have each call something like as.permutation() that handlesmultiple forms and converts to the one easiest for you to use. I am not sure that is needed as I suspect the simplest storage is something like a list: [0,3,2,4,5,6,7,1,9,8] but could also be shown with each cycle as a sub-list or something like anumpy vector or a customized class. Clearly if you control the package and how it is used, errors from bad data may not be a concern. But like many Boolean return(s) it is always a problem how to deal with a third possibility. -Original Message- From: Michael F. Stemper To: python-list@python.org Sent: Sat, Apr 23, 2022 8:57 am Subject: Re: Style for docstring On 22/04/2022 21.58, Avi Gross wrote: > Python does have a concept of "truthy" that includes meaning for not just the > standard Booleans but for 0 and non-zero and the empty string and many more > odd things such as an object that defines __bool__ (). > But saying it returns a Boolean True/False valuesounds direct and simple and > informative enough if that is True. > What bothers me is the assumption that anyone knows not so muchjust group > theory but what the argument to the function looks like as a Python object > of some kind. > Does the function accept only some permutation object managed by a specific > module? Will it accept some alternate representation such as a list structure > or other iterator? That's a fair point. However, this function will be the 22nd one in a module for dealing with permutations and groups of permutations. The module has a lengthy docstring explaining the several ways provided to specify a permutation. That way, the same information doesn't need to be written twenty-plus times. > Obviously deeper details would normally be in a manual page or other > documentation but as "permutations" are likely not to be what most people > think about before breakfast, or even after, odd as that may seem, ... I see what you did there :-> -- Michael F. Stemper Psalm 94:3-6 -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
I have been getting confused by how many interpretations and conditions for chasing tail people seem to be talking about. A fairly normal task is to want to see just the last N lines of a text-based file. A variant is the "tail -f" command from UNIX that continues to follow a growing file, often into a pipeline for further processing. The variant now being mentioned is a sort of "reverse" that has nothing to do with that kind of "tail" except if the implementation is to read the file backwards. A very straightforward way to reverse a file takes perhaps two lines of Python code by reading forward to fill a list with lines of text then using an index that reverses it. The issues being considered are memory and whether to read the entire file. I would think reading a file forwards in big chunks to be far faster and simpler than various schemes mentioned here for reading it backwards. It only makes sense if the goal is not reversal of all the contents. Also noted is that memory use can be minimized various ways so that only thefinal results are kept around. And if you really want more random access to files that you view as being organized as lines of text with a fixed or maximum width,then storing in some database format, perhaps indexed, may be a way to go. A time stamped log file is a good example. So which problem is really supposed to be solved for the original question? -Original Message- From: Roel Schroeven To: python-list@python.org Sent: Sun, Apr 24, 2022 5:19 am Subject: Re: tail dn schreef op 24/04/2022 om 0:04: > Disagreeing with @Chris in the sense that I use tail very frequently, > and usually in the context of server logs - but I'm talking about the > Linux implementation, not Python code! If I understand Marco correctly, what he want is to read the lines from bottom to top, i.e. tac instead of tail, despite his subject. I use tail very frequently too, but tac is something I almost never use. -- "Peace cannot be kept by force. It can only be achieved through understanding." -- Albert Einstein -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
Yes, Michael, a dictionary is an excellent way to represent a closed set of
transitions which your permutations are.
You examples use numerals but obviously a dictionary will allow transformations
of anything that can be hashed which mostly is items that are not mutable.
Of course for the purposes you may be using these permutations, accessing them
may be done carefully as you need to have nothing else in the dictionary and
must access all the keys and make sure you know which keys have already been
visited from another item.
Some other data structures may work better or faster on smaller examples.
I think you have satisfied my curiosity and your main and only question really
was on suggested wording of a Docstring.
Now if only the Docstring idea was replaced by a Dictionary too! Things like:
Dictstring = {"Purpose": "Text", "Args": "Text", "Return(s)": "Text",
"Optional-Note": "Text", "French version": DocStringFrench}
Too late to seriously change the language now!
-Original Message-
From: Michael F. Stemper
To: python-list@python.org
Sent: Sun, Apr 24, 2022 9:24 am
Subject: Re: Style for docstring
On 23/04/2022 12.43, Avi Gross wrote:
> Given what you added, Michael, your function is part of a larger collection
> of functions and being compatible with the others is a valid consideration.
> Whatever you decide, would ideally be done consistently with all or most of
> them.
> And, of course, it others in the collection also can handle multiple ways to
> specify a permutation, it may be simpler to have each call something like
> as.permutation() that handlesmultiple forms and converts to the one easiest
> for you to use.
> I am not sure that is needed as I suspect the simplest storage is something
> like a list: [0,3,2,4,5,6,7,1,9,8] but could also be shown with each cycle
> as a sub-list or something like anumpy vector or a customized class.
Since you ask, I'm using dictionaries as the internal representation.
If you think about it, a python dictionary *is* a function from one
finite set to another, mathematically. And a (finite) permutation is
a bijection from a (finite) set to itself.
For convenience, the module provides two methods of defining a permutation
other than just entering a dictionary:
>>> import PermGroups as pg
>>> a = {'1':'2', '2':'1', '3':'3'}
>>> b = pg.ParsePerm( '(12)(3)' )
>>> c = pg.ParseDomImg( '123', '213' )
>>> a==b
True
>>> b==c
True
>>>
All of the other functions work on these dictionaries.
I had thought about defining a permutation object, but the conceptual
match between "dict" and "permutation" was too good to discard.
> Clearly if you control the package and how it is used, errors from bad data
> may not be a concern.
An invalidly-constructed permutation will cause an exception, so
the function won't return.
>>> d = {'1':'2', '2':'2', '3':'3'}
>>> pg.ValidateDict(d)
False
>>>
If I was to do it over, I would have named this function something
like IsValidPermutation(), hiding the internal representation as
well as making the function's Boolean nature explicit.
--
Michael F. Stemper
No animals were harmed in the composition of this message.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
https://wiki.python.org/moin/PythonBooks -Original Message- From: Patrick 0511 To: python-list@python.org Sent: Wed, May 4, 2022 9:36 pm Subject: Python/New/Learn Hello, I'm completely new here and don't know anything about python. Can someone tell me how best to start? So what things should I learn first? -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
Chris, It was an extremely open-ended question to a forum where most of the readers are more advanced, at least I think. My library has oodles of Python Books for free to borrow on paper and return and I have read many of them. There are various e-books too, and of course lots of free internet resources including videos and on-line courses. If he wants simpler books, the web pages pointed here too: https://wiki.python.org/moin/IntroductoryBooks Next time, I won't try to be helpful and brief and just be silent. -Original Message- From: Chris Angelico To: python-list@python.org Sent: Wed, May 4, 2022 11:02 pm Subject: Re: Python/New/Learn On Thu, 5 May 2022 at 12:57, Avi Gross via Python-list wrote: > > https://wiki.python.org/moin/PythonBooks > That's an incredibly daunting list, and not something I'd overly strongly recommend, but yes, if you want to get a dead-tree or e-book to read, there are quite a lot of options available. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
I agree Chris that the Ukrainian Python Books are daunting as I barely started learning that language now even though my early years were just a few miles away and I might even have relatives still there! But as has been pointed out, suggestions are more helpful if you know a bit more about the one asking the questions or it is too broad. I do not feel qualified to suggest a book to true beginners as I leaned Python many decades after learning so many other computer languages and read a wide variety of books including many that assumed you already knew C or R or other languages, as I did. Someone new to programming may fin some resources including a tutorial handy. Someone who actually wants to use various modules like scikit-learn or pandas, or to solve specific problems, might well want to go straight to other resources or realize they need multiple resources over time. It can be a diversion to get someone to learn the functional programming aspects or even object-oriented if the goal is to do simple procedural things. Python is an extremely rich language and I recall reading a three-volume encyclopedia that took months as the books got pulled away by other readers. So, no Mark Lutz got a bit advanced. Oddly, I am learning Julia now and cannot find a single book in my group of Libraries, not even for the beginner I am not! But finding online resources was easy and if I have any reason to, plenty of books can be bought. It does not work for everybody, but my personal method is to attack issues and problems from multiple directions as each tends to reinforce and add to my knowledge. And I really appreciate when they tell you specifically how the language is different from others you may know so you know when not to expect it to work. A Julia documentation as an example, has a long list of places it is not like Python, or R or C++ and so on. If the person has computer language experience, some such resource might be better than something spending a thousand pages to teach from scratch. But, I am not volunteering to do personal tutoring. I prefer responding to specific focused questions especially after the person seems to have done some searching and research and reading on their own and maybe even shares some code and asks what may be wrong with it or ... And my first question would be why they chose to ask about Python. Was it their choice or required for a course or job or ... Sometimes the answer is to use something else they already know, albeit Python is a very decent language for many uses and well-worth learning even if you know others. -Original Message- From: Chris Angelico To: Avi Gross Cc: python-list@python.org Sent: Wed, May 4, 2022 11:21 pm Subject: Re: Python/New/Learn On Thu, 5 May 2022 at 13:14, Avi Gross wrote: > > Chris, > > It was an extremely open-ended question to a forum where > most of the readers are more advanced, at least I think. > > > My library has oodles of Python Books for free to borrow on paper and > return and I have read many of them. There are various e-books too, and > of course lots of free internet resources including videos and on-line > courses. > > > If he wants simpler books, the web pages pointed here too: > > > https://wiki.python.org/moin/IntroductoryBooks > > > Next time, I won't try to be helpful and brief and just be silent. > Being helpful is great, it's just that being brief can leave it as an incredibly scary-looking list :) If you want to recommend a couple of specific books, I think that would be a lot more helpful. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
Before more people reply to this user, I note I have not seen them reply back to the list about any questions or comments others have taken the time to provide. My warning bells go off when I see patterns and there was a similar request from another gmail account to an R language forum I am also on. They wanted help in learning programming (especially R) and claimed not to have any source to study. As we keep pointing out, you can trivially find sources including many free ones. So I wonder if there is point being sucked in by one or more people who don't even continue a conversation and perhaps are not even listening but just playing us to make us waste time. Or, maybe they are asking on multiple places to decide which to choose and are not saying so. Am I paranoid? Nah! But yes, a bit wary. I get so many kinds of SPAM in mail and phone calls and lately keep getting calls asking if I want to sell my house ... -Original Message- From: Schachner, Joseph To: Patrick 0511 ; python-list@python.org Sent: Thu, May 5, 2022 12:04 pm Subject: RE: Python/New/Learn Buy the book "Python 101" and do the examples. When you're done with that buy the book "Python 201" and study it. There is much more than is in both those books that you could learn about Python, but that's a very good way to start. --- Joseph S. Teledyne Confidential; Commercially Sensitive Business Data -Original Message- From: Patrick 0511 Sent: Wednesday, May 4, 2022 9:36 PM To: python-list@python.org Subject: Python/New/Learn Hello, I'm completely new here and don't know anything about python. Can someone tell me how best to start? So what things should I learn first? -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
This topic has rapidly shifted over way beyond Python even as the original person has not returned to participate. There are many ways to teach anything and since the classical method was to learn in person from someone using mainly sound or pantomime, it has hung on. Even with the existence of writing, books were made one at a time and were rare. In more recent times, the norm shifted gradually from lectures to individuals and groups to include textbooks and illustrations and eventually recordings or PowerPoint Slides with animation. Realistically, learning some things on your own is easy enough but for many people and many subjects, you need interaction, reinforcement and more. We have college professors who repeat almost the same identical lectures for years to various audiences and also take few or no questions. they might as well be recorded and find something better to do. But how do you learn French just from a book when it is completely not obvious how to pronounce anything given the weird spelling and grammar rules? How do you know if the dialect you use swallows some sounds or stretches them out a bit? For this you need to hear and perhaps see native speakers and correlate those sounds to the written words and learn to recognize and make them a habit. Even better, you often want someone to listen to what you try to say and respond and help guide you. Many Computer topics have an interesting side in that access to a computer running whatever you are learning can give you much experience and guidance as you can try various things and see how they work. Written text alone may be enough to learn what is special about a language and a set of problems to work on (or your own exploration) may be able to replace much of human interaction. You can look at learning systems such as COURSERA where they often break a "class" into parts that can include often shorter video clips often with subtitles or transcripts alongside it, as well as various kinds of printed material including tests and assignments and even ways (in some programming courses) to write small programs that are evaluated immediately by running them through the language program, or by having others (sometimes fellow students) grade them and return the results to you. There are many ideas out there how to learn. One of the worst is huge lecture halls with no rewind ... But text-only learning tools vary quite a bit and some of the better ones do not just throw facts at you but stop periodically and give you an overview of the goals and maybe add a touch of history that provides context on why some innovation was such a big improvement over what had been done and help you pronounce things when it is not obvious by saying that many people say a function name to rhyme with this or ... I used to hate Math textbooks that used every imaginable symbol and assumed you knew how to say every Greek letter and script L and integral symbol and an assortment of braces and brackets in various sizes and much more. It is hard to memorize formulas where you call lots of items by the name of "squiggle"! Python currently sticks largely to using standard ASCII characters so it has fewer issues to deal with. For people who are not native English speakers, though, some things may not be intuitively obvious, let alone pronounceable. I suspect for some purposes, a few lectures to listen to might help if well-designed. But I noticed how in Julia, they allow all kinds of symbols but also provide a way to make them fairly easily. Still their use of an actual lower-case epsilon as a synonym for "in" is an example of how teaching Julia may need more thantext for some people. It uses lots of unusual symbols for operators too thatare often familiar to mathematicians and hardly anyone else. for i ∈ 1:10 -Original Message- From: 2qdxy4rzwzuui...@potatochowder.com To: python-list@python.org Sent: Fri, May 6, 2022 8:56 am Subject: Re: Python/New/Learn On 2022-05-05 at 16:51:49 -0700, Grant Edwards wrote: > On 2022-05-05, Mats Wichmann wrote: > > > Without having any data at all on it, just my impressions, more > > people these days learn from in-person or video experiences. > > I've always been utterly baffled by video tutorials for > programming. There must be people who prefer that format, but it seems > like absolutely the worst possible option for me. You can't cut/paste > snippets from the examples. You have to constantly pause them so you > can try out examples. Sometimes it's not even easy to read the > examples. Perhaps if there was an accompanying web page or PDF... +1 (maybe more), except that an accompanying web page or PDF only solves the problem of copying/pasting examples badly, at the expense of the cognitive load to keep track of one more thing (because it's highly unlikely that the web page or PDF tracks the video "automatically"). As far as easy-to-read examples go, writing them down doesn't a
Re: tail
Marco, I think it was made clear from the start that "text" files in the classic sense have no random access method at any higher level than reading a byte at some offset from the beginning of the file, or back from the end when it has not grown. The obvious fact is that most of the time the lines are not of fixed widths and you have heard about multiple byte encodings and how the ends of lines can vary. When files get long enough that just reading them from the start as a whole, or even in chunks, gets too expensive, some people might consider some other method. Log files can go on for years so it is not uncommon to start a new one periodically and have a folder with many of them in some order. To get the last few lines simply means finding the last file and reading it, or if it is too short, getting the penultimate one too. And obviously a database or other structure might work better which might make each "line" a record and index them. But there are ways to create your own data that get around this such as using an encoding with a large but fixed width for every character albeit you need more storage space. But if the goal is a general purpose tool, internationalization from ASCII has created a challenge for lots of such tools. -Original Message- From: Marco Sulla To: Dennis Lee Bieber Cc: python-list@python.org Sent: Sat, May 7, 2022 9:21 am Subject: Re: tail On Sat, 7 May 2022 at 01:03, Dennis Lee Bieber wrote: > > Windows also uses for the EOL marker, but Python's I/O system > condenses that to just internally (for TEXT mode) -- so using the > length of a string so read to compute a file position may be off-by-one for > each EOL in the string. So there's no way to reliably read lines in reverse in text mode using seek and read, but the only option is readlines? -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
Just FYI, UNIX had a bunch of utilities that could emulate a vanilla version of tail on a command line. You can use sed, awk and quite a few others to simply show line N to the end of a file or other variations. Of course the way many things were done back then had less focus on efficiency than how to stepwise make changes in a pipeline so reading from the beginning to end was not an issue. -Original Message- From: Marco Sulla To: Chris Angelico Cc: python-list@python.org Sent: Wed, May 11, 2022 5:27 pm Subject: Re: tail On Wed, 11 May 2022 at 22:09, Chris Angelico wrote: > > Have you actually checked those three, or do you merely suppose them to be > true? I only suppose, as I said. I should do some benchmark and some other tests, and, frankly, I don't want to. I don't want to because I'm quite sure the implementation is fast, since it reads by chunks and cache them. I'm not sure it's 100% free of bugs, but the concept is very simple, since it simply mimics the *nix tail, so it should be reliable. > > > I'd very much like to see a CPython implementation of that function. It > > could be a method of a file object opened in binary mode, and *only* in > > binary mode. > > > > What do you think about it? > > Still not necessary. You can simply have it in your own toolkit. Why > should it be part of the core language? Why not? > How much benefit would it be > to anyone else? I suppose that every programmer, at least one time in its life, did a tail. > All the same assumptions are still there, so it still > isn't general It's general. It mimics the *nix tail. I can't think of a more general way to implement a tail. > I don't understand why this wants to be in the standard library. Well, the answer is really simple: I needed it and if I found it in the stdlib, I used it instead of writing the first horrible function. Furthermore, tail is such a useful tool that I suppose many others are interested, based on this quick Google search: https://www.google.com/search?q=python+tail A question on Stackoverflow really much voted, many other Stackoverflow questions, a package that seems to exactly do the same thing, that is mimic *nix tail, and a blog post about how to tail in Python. Furthermore, if you search python tail pypi, you can find a bunch of other packages: https://www.google.com/search?q=python+tail+pypi It seems the subject is quite popular, and I can't imagine otherwise. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
This seems to be a regular refrain where someone wants something as STANDARD in a programming language or environment and others want to keep it lean and mean or do not see THIS suggestion as particularly important or useful. Looking at the end of something is extremely common. Packages like numpy/pandas in Python often provide functions with names like head or tail as do other languages where data structures with names like data.frame are commonly used. These structures are in some way indexed to make it easy to jump towards the end. Text files are not. Efficiency aside, a 3-year-old (well, certainly a 30 year old) can cobble together a function that takes a filename assumed to be textual and reads the file into some data structure that stores the lines of the file and so it can be indexed by line number and also report the index of the final line. The data structure can be a list of lists or a dictionary with line numbers as keys or a numpy ... So the need for this functionality seems obvious but then what about someone who wants a bunch of random lines from a file? Need we satisfy their wish to pick random offsets from the file and get the line in which the offset is in middle of or the one about to start? Would that even be random if line lengths vary? Text files were never designed to be used efficiently except for reading and writing and certainly not for something like sorting. Again, generally you can read in the darn file and perform the operation and free up whatever memory you do not need. If you have huge files, fine, but then why make a special function be part of the default setup if it is rarely used? Why not put it in a module/package called BigFileBatches alongside other functions useful to do things in batches? Call that when needed but for smaller files, KISS. -Original Message- From: Dennis Lee Bieber To: python-list@python.org Sent: Wed, May 11, 2022 6:15 pm Subject: Re: tail On Thu, 12 May 2022 06:07:18 +1000, Chris Angelico declaimed the following: >I don't understand why this wants to be in the standard library. > Especially as any Linux distribution probably includes the compiled "tail" command, so this would only be of use on Windows. Under recent Windows, one has an equivalent to "tail" IFF using PowerShell rather than the "DOS" shell. https://www.middlewareinventory.com/blog/powershell-tail-file-windows-tail-command/ or install a Windows binary equivalent http://tailforwin32.sourceforge.net/ -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: .0 in name
Bryan,
As has been pointed out, it is very common in possibly all programming
languages to not allow digits at the start of many identifiers. It makes it
hard to parse for numbers which tend to start with digits. Some languages even
have special rules on not starting a number with a zero unless you mean for it
to be seen as octal (or 0x for hexadecimal) and many other rules exist.
There are languages where 12x means 12*x so even the lack of an operator ...
There are exceptions that often are not really exceptions. You can use all
kinds of unusual variables in some quoted context. It is valid (albeit not
encouraged) to use backquoted
The following is perfectly allowed in R:
> `5x^2 + 2.3x` <- 666 > `+-+-+` <- 1000 > 1 + 2 * `5x^2 + 2.3x` + `+-+-+` [1]
> 2333
And there are often issued when you do things like create the name of a column
of data in a data.frame with embedded spaces and other anomalies requiring
special handling.
So why you wonder where it is documented that variables cannot be what you feel
like is a bit puzzling!
-Original Message-
From: bryangan41
To: python-list@python.org
Sent: Fri, May 13, 2022 12:47 pm
Subject: .0 in name
May I know (1) why can the name start with a number?(2) where in the doc is
it?!>>> import pdb>>> pdb.run('(a for a in "")')> (1)()(Pdb)
s--Call--> (1)()(Pdb) a.0 = (Pdb) c>>>Sent from Samsung tablet.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: .0 in name
Boy do I hate when I see my code mangled by the stupid AOL mailer.
Not that anyone cares, but the code should be read with each line starting with
the "> " prompt.
If I leave lots of blank lines, it may work, but as the illustration is not in
python, I will now remove the prompts:
`5x^2 + 2.3x` <- 666
`+-+-+` <- 1000
1 + 2 * `5x^2 + 2.3x` + `+-+-+`
output: 2333
There are rarely good reasons for such silly variable names but as long as you
let it know to leave the quoted regions alone when parsing and look them us as
exact entries in various environments, it works fine.
To add to what others already wrote, many languages have serious requirements
you need to be aware of and not assume otherwise. Some allow underscores in
names and may limit that in the first part or may in some cases suggest or
require it. Some have rules about whether a variable of one kind should start
with an uppercase letter. Some allow periods in names although an initial
period may make it invisible for some purposes. And, some newer languages allow
all kinds of UNICODE characters and perhaps even some that can be seen as
numeric but aren't exactly 0-9.
① ② ③
... ❽ ❽
-Original Message-----
From: Avi Gross via Python-list
To: python-list@python.org
Sent: Fri, May 13, 2022 6:02 pm
Subject: Re: .0 in name
Bryan,
As has been pointed out, it is very common in possibly all programming
languages to not allow digits at the start of many identifiers. It makes it
hard to parse for numbers which tend to start with digits. Some languages even
have special rules on not starting a number with a zero unless you mean for it
to be seen as octal (or 0x for hexadecimal) and many other rules exist.
There are languages where 12x means 12*x so even the lack of an operator ...
There are exceptions that often are not really exceptions. You can use all
kinds of unusual variables in some quoted context. It is valid (albeit not
encouraged) to use backquoted
The following is perfectly allowed in R:
> `5x^2 + 2.3x` <- 666 > `+-+-+` <- 1000 > 1 + 2 * `5x^2 + 2.3x` + `+-+-+` [1]
> 2333
And there are often issued when you do things like create the name of a column
of data in a data.frame with embedded spaces and other anomalies requiring
special handling.
So why you wonder where it is documented that variables cannot be what you feel
like is a bit puzzling!
-Original Message-
From: bryangan41
To: python-list@python.org
Sent: Fri, May 13, 2022 12:47 pm
Subject: .0 in name
May I know (1) why can the name start with a number?(2) where in the doc is
it?!>>> import pdb>>> pdb.run('(a for a in "")')> (1)()(Pdb)
s--Call--> (1)()(Pdb) a.0 = (Pdb) c>>>Sent from Samsung tablet.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: .0 in name
You left out 3CPO, Dave. Names with numerals are not unreasonable in some circumstances. But programming languages are not a full spectrum of real life. They can have reserved words too so you may not be able to use "while" and if you choose to create a function that masks another, you cannot complain that you assumed the computer would be smart enough to figure out which one you wanted. Yes, some languages encourage multiple functions with the same name but different signatures. Bottom line is each has RULES and some serious SUGGESTIONS. It is what it is, not what you want it to be. But although starting with a numeral is verboten for variable names, may I suggest that a relatively invisible character can make a name like _3CPO that will be allowed. But be careful as Python programs often have conventions on use of the underscore and don't even think about a dundering name like __init__ ... -Original Message- From: dn To: python-list@python.org Sent: Sat, May 14, 2022 12:33 am Subject: Re: .0 in name This is not what @Avi menat by "silly variable names" but: 3D_position 2nd_floor_area 3M_PostIt_size 3rd_degree_polynomial 360_degree_view 12_hours_later and ??? 2_fast_2_furious -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: oop issue
invest_crypto.client_list.append(self)
I am wondering about the phrasing above.
When you are in the dunder init function, you normally create and change items
in YOURSELF so why is your code not changing self.crypto_client_list?
And what are you appending to before ever creating it? Would it kill you to
create some form of container in the main class definition initialized
appropriately to some variation of empty?
I won't claim to fully understand what the code wants to do. Adding yourself to
the list of clients may make sense to you but if you understand object oriented
programming, you may have an inkling that objects need to be CREATED somewhere
before they can be used. Most of your code looks like it is DEFINING an object.
The last few lines try to access a method in an object that has never been
instantiated. Yes, you do have a way to store methods in a class and call them
without any objects but this is not a case like that.
You need something like "myobj = invest_crypto(args)" and then the rest of your
code can do changes and calculations and perhaps create other similar or
different objects. You seem to be using a method that reads in a file and dumps
a version of the contents and that might work if you did a line like this next:
"myobj.access_client_details()" albeit not how I would name it or do it.
And, of course, your print() again names the class, not an instance of a class.
Your code wraps in a few places and I wonder if it contains errors as in this
set of lines:
return f"('{self.name}', '{self.surname}', '{self.amount_Deposited}',
'{self.amount_to_transfer}')"
Is that really how you think you set up a formatted string?
Even if you get that working, how does the print statement know what to do with
a list of objects?
Some might design a method you can call to print the contents of a list which
would loop over the list. But is there ever more than one client (yourself) in
the list?
The above rambling is just reflecting my opinion that you have not learned
enough or thought it through and may even be copying and modifying various
snippets of code perhaps from places where it works to a place that might be
better designed from scratch.
Have fun. As someone else mentioned, smaller more focused examples may work
better to get you up to speed, but then again, we often find out someone is
given a homework assignment ...
-Original Message-
From: Tola Oj
To: python-list@python.org
Sent: Mon, May 23, 2022 4:54 pm
Subject: oop issue
i just finished learning oop as a beginner and trying to practice with it
but i ran into this typeerror issue, help please.
Traceback (most recent call last):
File
"c:\Users\ojomo\OneDrive\Desktop\myexcel\oop_learn.py\myExperiment.py\mainMain.py",
line 36, in
print(invest_crypto.client_list)
TypeError: invest_crypto.__repr__() missing 1 required positional argument:
'self'
this is my code below:
import csv
class invest_crypto:
crypto_current_rate = 0.05
client_list = []
def __init__(self, name, surname, amount_Deposited, amount_to_transfer):
self.name = name
self.surname = surname
self.amount_Deposited = amount_Deposited
self.amount_to_transfer = amount_to_transfer
invest_crypto.client_list.append(self)
def calculate_customer_transfer(self):
self.customer_transfer = (self.crypto_current_rate * self.
amount_Deposited) + self.amount_Deposited
return self.customer_transfer
@classmethod
def access_client_details(cls):
with open('C:\\Users\\ojomo\\OneDrive\\Desktop\\myexcel\\
oop_learn.py\\myExperiment.py\\clientDetails.csv', 'r' ) as f:
reader = csv.DictReader(f)
clientDetails = list(reader)
for item in clientDetails:
invest_crypto(
name=item.get('name'),
surname=item.get('surname'),
amount_Deposited=item.get('amount_deposited'),
amount_to_transfer=item.get('amount_to_transfer')
)
@staticmethod
def __repr__(self):
return f"('{self.name}', '{self.surname}', '{self.amount_Deposited}',
'{self.amount_to_transfer}')"
invest_crypto.access_client_details()
print(invest_crypto.client_list)
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to test characters of a string
Amazing how some people bring out the heavy artillery, first! LOL! If the question was how to remove any initial digits and perhaps whitespace in a string, it is fairly easy to do without any functions to test if there are digits before the title. I mean look at initial characters and move forward if it is between '0' and '9' or a space. Duh! Sure, a regular expression that matches anything following a run of digits and whitespace and before a ".MPG" or the end of the entry will be easy to extract and compare after removing any left/right whitespace in both things being compared and coercing both to the same case. But the solution may be doomed to failure when it sees things like: "100 Letters" "1+1" "10,000 hours" "1 Trillion Dollar$" "2,000 Light Years From Home" So is it necessary to insist on an exact pattern of two digits followed by a space? That would fail on "44 Minutes", "40 Oz. Dream", "50 Mission Cap", "50 Ways to Say Goodbye", "99 Ways to Die" It looks to me like you need to compare TWICE just in case. If it matches in the original (perhaps with some normalization of case and whitespace, fine. If not will they match if one or both have something to remove as a prefix such as "02 ". And if you are comparing items where the same song is in two different numeric sequences on different disks, ... -Original Message- From: Christian Gollwitzer To: python-list@python.org Sent: Tue, Jun 7, 2022 6:01 pm Subject: Re: How to test characters of a string Am 07.06.22 um 21:56 schrieb Dave: > It depends on the language I’m using, in Objective C, I’d use isNumeric, just > wanted to know what the equivalent is in Python. > Your problem is also a typical case for regular expressions. You can create an expression for "starts with any number of digits plus optional whitespace" and then replace this with nothing: > chris@linux-tb9f:~> ipython > Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] > Type 'copyright', 'credits' or 'license' for more information > IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help. > > In [1]: import re > > > > In [2]: s='05 Trinket' > > > > In [3]: re.sub(r'^\d+\s*', '', s) > > > Out[3]: 'Trinket' > If it doesn't match, it will do nothing: > In [4]: s='Es geht los' > > > > In [5]: re.sub(r'^\d+\s*', '', s) > > > Out[5]: 'Es geht los' Some people on this list don't like regexes but for tasks like this they are made and working well. ^ is "starts with" \d is any digit \s is any space + is at least one * is nothing or one of Christian -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: How to replace characters in a string?
Dave,
Your goal is to compare titles and there can be endless replacements needed if
you allow the text to contain anything but ASCII.
Have you considered stripping out things instead? I mean remove lots of stuff
that is not ASCII in the first place and perhaps also remove lots of extra
punctuation likesingle quotes or question marks or redundant white space and
compare the sort of skeletons of the two?
And even if that fails, could you have a measure of how different they are and
tolerate if they were say off by one letter albeit "My desert" matching "My
Dessert" might not be a valid match with one being a song about an arid
environment and the other about food you don't need!
Your seemingly simple need can expand into a fairly complex project. There may
be many ideas on how to deal with it but not anything perfect enough to catch
all cases as even a trained human may have to make decisions at times and not
match what other humans do. We have examples like the TV show "NUMB3RS" that
used a perfectly valid digit 3 to stand for an "E" but yet is often written
when I look it up as NUMBERS. You have obvious cases where titles of songs may
contain composite symbols like "œ" which will not compare to one where it is
written out as "oe" so the idea of comparing is quite complex and the best you
might do is heuristic.
UNICODE has many symbols that are almost the same or even look the same or
maybe in one font versus another. There are libraries of functions that allow
some kinds of comparisons or conversions that you could look into but the gain
for you may not be worth it. Nothing stops a person from naming a song any way
they want and I speak many languages and often see a song re-titled in the
local language and using the local alphabet mixed often with another.
Your original question is perhaps now many questions, depending on what you
choose. You started by wanting to know how to compare and it is moving on to
how to delete parts or make substitutions or use regular expressions and it can
get worse. You can, for example, take a string and identify the words within it
and create a regular expression that inserts sequences between the words that
match any zero or one or more non-word characters such as spaces, tabs,
punctuation or non-ASCII, so that song titles with the same words in a sequence
match no matter what is between them. The possibilities are endless but
consider some of the techniques that are used by some programs that parse text
and suggest alternate spellings or even programs like Google Translate that
can take a sentence and then suggest you may mean a slightly altered sentence
with one word changed to fit better.
You need to decide what you want to deal with and what will be mis-classified
by your program. Some of us have suggested folding the case of the words but
that means asong about a dark skinned person in Poland called "Black Polish"
would match a song about keeping your shoes dark with "black polish" so I keep
repeating it is very hard or frankly impossible, to catch every case I can
imagine and the many I can't!
But the emphasis here is not your overall problem. It is about whether and how
the computer language called python, and perhaps some add-on modules, can be
used to solve each smaller need such as recognizing a pattern or replacing
text. It can do quite a bit but only when the specification of the problem is
exact.
-Original Message-
From: Dave
To: python-list@python.org
Sent: Wed, Jun 8, 2022 5:09 am
Subject: Re: How to replace characters in a string?
Hi,
Thanks for this!
So, is there a copy function/method that returns a MutableString like in
objective-C? I’ve solved this problems before in a number of languages like
Objective-C and AppleScript.
Basically there is a set of common characters that need “normalizing” and I
have a method that replaces them in a string, so:
myString = [myString normalizeCharacters];
Would return a new string with all the “common” replacements applied.
Since the following gives an error :
myString = 'Hello'
myNewstring = myString.replace(myString,'e','a’)
TypeError: 'str' object cannot be interpreted as an integer
I can’t see of a way to do this in Python?
All the Best
Dave
> On 8 Jun 2022, at 10:14, Chris Angelico wrote:
>
> On Wed, 8 Jun 2022 at 18:12, Dave wrote:
>
>> I tried the but it doesn’t seem to work?
>> myCompareFile1 = ascii(myTitleName)
>> myCompareFile1.replace("\u2019", "'")
>
> Strings in Python are immutable. When you call ascii(), you get back a
> new string, but it's one that has actual backslashes and such in it.
> (You probably don't need this step, other than for debugging; check
> the string by printing out the ASCII version of it, but stick to the
> original for actual processing.) The same is true of the replace()
> method; it doesn't change the string, it returns a new string.
>
word = "spam"
print(word.replace("sp", "h"))
>
Re: How to test characters of a string
Dave, Sometimes a task is done faster by NOT programming anything in any language! Not only have you spent a lot of your own time but many dozens of messages here have dragged in others, who gain nothing ;-) The domain you are operating in seems to have lots of variants in how the titles are stored as names and you keep finding new variants. Yes, if your goal is to use this as a way to learn more in general about Python, clearly it may meet that goal! Contrary to what I (and some others) said earlier, it may be time to consider regular expressions and other heavier artillery! LOL! I do not plan on sticking with your twists and turns but will quickly address your variants. Sides that come in two's like records are presumably limited to using A and B in your example. But multiple disks connected can mean the digit(s) following can have double digits or even more. Your python code may have to contain lots of functions you create that match some pattern and return some value and perhaps other functions that know how to convert from that format to a common canonical format of your own so they can be compared. Your main code may need to try them in various sequences till it finds a match and so on. But when you are done, in what format do you save them? The original or your minimal? Still confusing to me, as someone who does not give a darn, is the reality that many songs may have the same name but be different as in a song from Sinatra when he was young and a later recording with a different orchestra or by a Sinatra imitator. They may all be titled something like "New York, New York" or "NEW YORK -- NEW YORK" which your algorithm folds into the same characters. So I am guessing you also need to access other data about the person who sings it or what year it was released to make comparisons. At some point you may want to create or borrow some sort of class/object that encapsulates your data as well as methods that let you do things like make a canonical version of the Title and then a way to ask if Object A is reasonably equal to object B might happen if you define a function/method of __eq__ for that class. It might take you years and need periodic refining as you encounter ever more varied ways people have chosen to label their music, but so what? LOL! Humor or sarcasm aside, your incremental exploratory method reminds me why it is a good idea to first scope out the outlines of your problem space and make some key decisions and write out a fairly detailed set of requirements before seriously making more than prototypes. You might get very different help from people if they understood that your first request was far from complete but only one of many that may better be worked on some other way. And I wonder if you did any search of the internet to see if anyone had done anything similar in Python (or another language) that may handle parts of what you need before asking here. I note lots of people who come with what they consider a good programming background have to adjust to aspects of languages like python as what they know is in some ways wrong or inadequate in a new environment. -Original Message- From: Dave To: python-list@python.org Sent: Thu, Jun 9, 2022 2:50 am Subject: Re: How to test characters of a string Hi, I’ve found you also need to take care of multiple disk CD releases. These have a format of “1-01 Track Name” “2-02 Trackl Name" Meaning Disk 1 Track1, Disk 2, Track 2. Also A and B Sides (from Vinyl LPs) “A1-Track Name” “B2-Track Name” Side A, Track 1, etc. Cheers Dave > On 8 Jun 2022, at 19:36, Dennis Lee Bieber wrote: > > On Wed, 8 Jun 2022 01:53:26 + (UTC), Avi Gross > declaimed the following: > > >> >> So is it necessary to insist on an exact pattern of two digits followed by a >> space? >> >> >> That would fail on "44 Minutes", "40 Oz. Dream", "50 Mission Cap", "50 Ways >> to Say Goodbye", "99 Ways to Die" >> >> It looks to me like you need to compare TWICE just in case. If it matches in >> the original (perhaps with some normalization of case and whitespace, fine. >> If not will they match if one or both have something to remove as a prefix >> such as "02 ". And if you are comparing items where the same song is in two >> different numeric sequences on different disks, ... > > I suspect the OP really needs to extract the /track number/ from the > ID3 information, and (converting to a 2digit formatted string) see if the > file name begins with that track number... The format of the those > filenames appear to be those generated by some software when ripping CDs to > MP3s -- for example: > > -=-=- > c:\Music\Roger Miller\All Time Greatest Hits>dir > Volume in drive C is OS > Volume Serial Number is 4ACC-3CB4 > > Directory of c:\Music\Roger Miller\All Time Greatest Hits > > 04/11/2022 05:06 PM . > 04/11/2022 05:06 PM .. > 07/26/2018 11:20 AM 4,493,279 01 Dang
Re: Function to Print a nicely formatted Dictionary or List?
Dave, Despite your other programming knowledge, I suspect you think this is the forum where people come to be tutored. Try here: https://mail.python.org/mailman/listinfo/tutor Yes, there are plenty of pretty printers available and you can build your own function fairly easily. A module like pprint may have what you want in pprint.pprint() but you can write a function for yourself that takes a dictionary and loops through items and prints them one per line and, if you feel like it, also prints how many items there are and your own custom touches such as doing them alphabetically. Consider using a search engine before posting. Throw in a few words like "python pretty print dictionary function" and refine that if it does not get you immediate results. It is free and easy and does not waste time for so many others who already know or don't care. And consider reading a few books perhaps designed to teach python to people with some programming experience or taking a course on something like COURSERA as a part of your learning process and not depending on volunteers so much. Much of what you are asking is covered in fairly beginner and intermediate such books/courses. I think I am now going to ignore messages from you for a while. Signal to noise ratio ... -Original Message- From: Dave To: python-list@python.org Sent: Thu, Jun 9, 2022 6:43 am Subject: Function to Print a nicely formatted Dictionary or List? Hi, Before I write my own I wondering if anyone knows of a function that will print a nicely formatted dictionary? By nicely formatted I mean not all on one line! Cheers Dave -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: fill out bulletins
I wish this discussion was simplified. It sounds to me like what is wanted is a way to PRINT a filled-out form using some dynamic text that fits over designated slots in the data. It is not that different from many other tasks where you overlay some graphics with text. You need a decent version of the graphics with enough pixels and the right size using something as simple as a scanner at higher resolution OR getting someone to give you a template in a form you can use. Using a ruler or other tools, map out the rectangles you are going to be printing text in. Use your own algorithms to take each snippet of text you want to print and do wrapping and resizing of the font or whatever it takes to make it fit. Finally, overlay the text snippets on the image. Then PRINT it on a decent printer. From what I hear, you seem not to need the original text anymore, but nothing stops your program with storing bits you will use again later and reusing them, or having a log of the original text used and so on. Plan B might be to COPY your image on a ream of paper and then replace the paper in the printer so the next print writes on top of it. Create a template with just text that when printed will happen to write over the same spots. As to what tools you can use, there are many to choose from. You asked on a Python list so you may want some of the Python Graphics utilities. In R, I might use ggplot which lets me set a background layer then draw objects above it at various offsets, as one of many things. I know many businesses do things like this all the time such as having printers loaded with checks to be printed, or inserting an envelope into a slot on the printer to have the name and address printedin the right places. -Original Message- From: Peter Pearson To: python-list@python.org Sent: Tue, Jun 14, 2022 9:28 am Subject: Re: fill out bulletins On Tue, 14 Jun 2022 00:41:07 +0200, jak wrote: [snip] > > If you are interested in seeing what I called "post office bulletin" > (English is not my language and I don't know the name, sorry), you can > find a sample pdf (fillable) but it works badly here: > > https://www.guardiacostiera.gov.it/venezia/Documents/Bollettino%20MOD.%20TD123.pdf Are these "post office bulletins" always PDFs? -- To email me, substitute nowhere->runbox, invalid->com. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: "CPython"
This leads to the extremely important question of what would an implementation of Python, written completely in C++, be called? C++Python CPython++ C+Python+ DPython SeaPython? SeeSeaSiPython I don't even want to think fo what sound a C# Python would make. OK, my apologies to all. Being an interpreted language, it makes sense for a good part of the interpreter to include parts made in other languages and also add-on libraries in even older languages like FORTRAN. Quite a few languages, including some like R, are also partially based on C in similar ways. -Original Message- From: Paulo da Silva To: python-list@python.org Sent: Mon, Jun 20, 2022 8:53 pm Subject: Re: "CPython" Às 20:01 de 20/06/22, Paulo da Silva escreveu: > Às 18:19 de 20/06/22, Stefan Ram escreveu: >> The same personality traits that make people react >> to troll postings might make them spread unconfirmed >> ideas about the meaning of "C" in "CPython". >> >> The /core/ of CPython is written in C. >> >> CPython is the /canonical/ implementation of Python. >> >> The "C" in "CPython" stands for C. >> >> > > Not so "unconfirmed"! > Look at this article, I recently read: > https://www.analyticsinsight.net/cpython-to-step-over-javascript-in-developing-web-applications/ > > > > There is a sentence in ther that begins with "CPython, short for Core > Python, a reference implementation that other Python distributions are > derived from, ...". > > Anyway, I wrote "IMHO". > > Do you have any credible reference to your assertion "The "C" in > "CPython" stands for C."? > > Thank you. Well ... I read the responses and they are not touching the point! I just answered, with my opinion based on articles I have read in the past. Certainly I could not be sure. That's why I responded as an opinion (IMHO) and not as an assertion. Stefan Ram responded with a, at least, not very polite post. That's why I needed to somehow "defend" why I posted that response, and, BTW, trying to learn why he said that the C in CPython means "written in C". I still find very strange, to not say weird, that a compiler or interpreter has a name based in the language it was written. But, again, is just my opinion and nothing more. I rest my case. Thank you all. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: "CPython"
If we want to be humorous, RPython would obviously either be written in R, which really is not designed well for such purposes, or would be some kind of synthesis that already exists that allows you to run R and python code interchangeably on sort of shared data that I sometimes do in RSTUDIO. I like the way you think Greg. I did not consider how the ++ in C++ is a bit like stuttering and since python also starts with a P the effect would be something like C-p-p-python. My problem with that idea is, believe it or not, that it is too negative. What you meant to be seen as a dash is a minus sign to me. And both C and C++ not only have both a pre and post autoincrement variable using ++x and x++, they also have autodecrement operators using a minus sign such as --x and x-- and it can get pretty weird trying to figure out if some code is legal, let alone what it does, without parentheses. I mean what the heck does this do? y = x++-++x The truth is that although I remember Bjarne trying to figure out a good name for his somewhat improved language and his choice of C++ rather than D or some other gimmick, you could argue he also removed a bit from C. But who would call a language C-- ?? Back to serious. This discussion is more about names but is it? Some of the implementations of Python are not just written in some computer language but also in a sort of environment. Arguably some core of functionality has to be pretty much consistent to the language definition. But each may add interesting twists on its own and that can include the ability to easily link in code and libraries written in that language or used in that environment. You can have different supersets of a language. And it can impact where you might use the language as one reason people may not understand for using C is that a compiler was available for just about anywhere that either ran in that environment or could be run on another to produce lower-level code to copy to it. It was also possible to embed code in C that was evaluated differently in each environment for some level of fine-tuning. Some of that may eventually have been true for other implementations but I suspect not for some deliberately designed to fit what one party wants and with no care that it be shared elsewhere especially as the C version was already available there. Or am I wrong? After all, others who kept improving C thought the ++ concept was best removed! -Original Message- From: Greg Ewing To: python-list@python.org Sent: Tue, Jun 21, 2022 3:53 am Subject: Re: "CPython" On 21/06/22 2:56 pm, Paulo da Silva wrote: > Let's say they reimplement "reference python" CPython in Rust. What is > better? Change the "reference python" CPython name to RPython, for > example, or let it as CPython? The C implementation would still be called CPython, and the new implementation might be called RPython, or RustyPython, or whatever. The names are independent of which one is currently blessed as the reference implementation. Although if it were called RPython, no doubt a new debate would flare up over whether the "R" stands for "Rust" or "Reference"... -- Greg -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: "CPython"
David, I am curious why you are undertaking the effort to take a language already decades old and showing signs of being a tad rusty into a language that suggests further oxidation. More seriously, I am interested in what this can gain and the intended user base. I studied Rust for a while and it has it's features but have had no opportunity to use it. Is it expected to make a faster version of Python, or enable better connections to libraries and so on? What I mean is that if you are planning on making it pass all tests for python functionality, are you also adding unique features or ... ? My preference is to have names that fully include what they are about. So the name "python" would be left intact rather than mangled, even if the name itself happens to be totally meaningless. So may I suggest something like """rustic-python""" ? -Original Message- From: David J W To: python-list@python.org Sent: Thu, Jun 23, 2022 10:29 am Subject: Re: "CPython" >> Let's say they reimplement "reference python" CPython in Rust. What is >> better? Change the "reference python" CPython name to RPython, for >> example, or let it as CPython? >The C implementation would still be called CPython, and the new >implementation might be called RPython, or RustyPython, or whatever. >The names are independent of which one is currently blessed as the >reference implementation. I am at the pre planning stages of making a Rust implementation of the Python virtual machine and to avoid ambiguity I've been working with Rython as the name. I tried looking for a Monty Python themed name but the good ones seem to be taken. Otherwise as for a timeline, solo I figure it's going to take me a couple years to get something that actually passes cpython's python unit-tests. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: "CPython"
David, I understand now. As a project for your own edification I can understand it, albeit it is a more substantial effort than many people might choose, LOL! So unless it starts being used heavily and adopted by some organization, the result of your effort will not necessarily be compatible with many modules now available or keep up with changes as python adds features or fixes bugs. I am curious about why something like numpy could not be integrated into what you do. Of course, if you are the only user, ... My hobbies to spend my time may not be as ambitious, but are quite a bit more varied! LOL! -Original Message- From: David J W To: Avi Gross Cc: python-list@python.org Sent: Fri, Jun 24, 2022 11:57 am Subject: Re: "CPython" The main motivation for a Python virtual machine in Rust is to strengthen my knowledge with Rust which currently has some gnarly bits to it but otherwise is an impressive low level language. Rust's future is looking very bright as even Linus Torvalds agrees with most of its design choices and is allowing it to be used as a linux kernel module language. Skipping ahead to the subject of names, Rython was chosen because "Python" is trademarked by the PSF so anything with the complete word Python in it is out. A close runner up would have been Camelot but that is already taken. Going backward to the issue of use and audience. Making Rython a real virtual machine that passes the CPython unit-tests is the only goal. I am actively following the faster CPython fork that Mike Shannon, GVR, and others are working on with the intention to try and incorporate what they discover into my project but I don't think Rython will be dramatically faster than Cpython because I am going to implement the same PyObject reference counting garbage collector and unless faster CPython creates a JIT component, Rython won't have one either. Additionally Ryhon won't have the must have killer libraries like numpy so it's a moot point if my project turns out to be dramatically faster. To sum things up, I've been retired for over a decade so I have plenty of free time. Initially I thought I might invest time into becoming a core python developer but looking into it further, all I will say is that doesn't feel like a very appealing use of my time. On Thu, Jun 23, 2022 at 9:42 AM Avi Gross wrote: David, I am curious why you are undertaking the effort to take a language already decades old and showing signs of being a tad rusty into a language that suggests further oxidation. More seriously, I am interested in what this can gain and the intended user base. I studied Rust for a while and it has it's features but have had no opportunity to use it. Is it expected to make a faster version of Python, or enable better connections to libraries and so on? What I mean is that if you are planning on making it pass all tests for python functionality, are you also adding unique features or ... ? My preference is to have names that fully include what they are about. So the name "python" would be left intact rather than mangled, even if the name itself happens to be totally meaningless. So may I suggest something like """rustic-python""" ? -Original Message- From: David J W To: python-list@python.org Sent: Thu, Jun 23, 2022 10:29 am Subject: Re: "CPython" >> Let's say they reimplement "reference python" CPython in Rust. What is >> better? Change the "reference python" CPython name to RPython, for >> example, or let it as CPython? >The C implementation would still be called CPython, and the new >implementation might be called RPython, or RustyPython, or whatever. >The names are independent of which one is currently blessed as the >reference implementation. I am at the pre planning stages of making a Rust implementation of the Python virtual machine and to avoid ambiguity I've been working with Rython as the name. I tried looking for a Monty Python themed name but the good ones seem to be taken. Otherwise as for a timeline, solo I figure it's going to take me a couple years to get something that actually passes cpython's python unit-tests. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: NILEARN - WHY THIS CODE THROWS AN ERROR?????
Nati Stern has asked several questions here, often about relatively technical
uses of python code that many of us have never used and still is not providing
more exact info that tends to be needed before anyone can even think of
diagnosing the problem.
I have learned to stay away from some such questioners. But I am wondering if
some people (others too) think this forum is a generalized help desk staffed by
College Professors with nothing else to do.
Many questions are best handled locally where people can look over your
shoulder or use the same software and may have some fluency in your native
language. And sometimes you need to do more investigating on your own, and
perhaps tell us what you tried and why it was not useful, or we end up making
endless suggestions and being told we are not working on your real issue and so
on.
The code below is just babel or maybe babble. Something nested in a loop had a
problem. Why not try something drastic and look at the files and PICK ONE and
use it step by step and see when it fails?
It looks like the code wants to ask for all files then ignore some.
Why you would import numpy repeatedly in a loop is beyond me! LOL!
But which command line failed? My GUESS is:
data = img.get_fdata()
If so, did you try to see the current value of the filename you call "i" in the
loop and see what name was loaded in what looks like a file ending in .nii in
this code:
img = nib.load(path+"/"+i)
You need to proceed step by step and see if any previous steps failed.
But what is possible is you got a file with .nii in middle of the name that
does not end in .gz, or is not in the format needed.
Good luck,
אבי גרוס
-Original Message-
From: MRAB
To: python-list@python.org
Sent: Fri, Jul 8, 2022 4:47 pm
Subject: Re: NILEARN - WHY THIS CODE THROWS AN ERROR?
On 08/07/2022 14:06, נתי שטרן wrote:
> fullcode:
>
>
>
> import nilearn.plotting as plot
> import os,gzip,io
> import nibabel as nib
> path="C:/users/administrator/desktop/nii"
> path2="C:/users/administrator/desktop/nii/out/"
> for i in os.listdir(path):
> if(".nii.gz" in i):
> pass
> else:
>
> if(".nii" in i):
> img = nib.load(path+"/"+i)
> data = img.get_fdata()
> print(data)
> import imageio
> X=0
> for s in data:
> import numpy
> aleph=numpy.array(s,dtype=numpy.int8)
> X=X+1
> plot.plot_img(aleph)
>
> imageio.imwrite("C:\\users\\administrator\\desktop\\nii\\"+i
> +str(X)+'.jpg', s)
>
>
>
>
>
>
> error:
> Data given cannot be loaded because it is not compatible with nibabel
> format
>
What's the complete traceback?
It might help you to identify the problem if you add messages to tell
you what it's doing, e.g. what file it's about to load.
Apparently, one of the files is not compatible with nibabel.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: what's the problem??????
Nati,
This is a two-way process and requires you to be very clear on what is not
working or what you are trying to do or help clear away having us try to
understand lots of code that is not very related to the question.
Your code, as shown, makes an empty string repeatedly in a loop.
a=dict()
I am guessing the code there works fine and does nothing useful. Well, what do
you want in your dictionary? Most people create a dictionary outside the loop
as an empty dictionary and then in the loop use one of many methods to add
key:value pairs.
Earlier in the code you had another line:
a=labels.to_dict()
If the labels variable had a method like that, that is also a way.
So be specific about what LINE or region of code and what is wrong and what you
already tried or error messages you got.
Avi
-Original Message-
From: נתי שטרן
To: Neuroimaging analysis in Python ;
python-list@python.org
Sent: Wed, Jul 13, 2022 2:35 pm
Subject: Re: what's the problem??
I want to set dict
בתאריך יום ד׳, 13 ביולי 2022, 20:47, מאת נתי שטרן :
> CODE:
>
> for nii in os.listdir("c:/users/administrator/desktop/nii"):
>
> from nilearn import plotting
> from nilearn import datasets
> atlas = datasets.fetch_atlas_msdl()
> # Loading atlas image stored in 'maps'
> atlas_filename =
> "C:/Users/Administrator/Desktop/64/64/2mm/maps.nii.gz"
> # Loading atlas data stored in 'labels'
> labels = pd.read_csv(
> "C:/Users/Administrator/Desktop/64/64/labels_64_dictionary.csv")
> a=labels.to_dict()
> b=a["Difumo_names"]
> from nilearn.maskers import NiftiMapsMasker
> masker = NiftiMapsMasker(maps_img=atlas_filename, standardize=True,
> memory='nilearn_cache', verbose=5)
>
> time_series = masker.fit_transform(
> "c:/users/administrator/desktop/nii/"+nii)
> try:
> from sklearn.covariance import GraphicalLassoCV
> except ImportError:
> # for Scitkit-Learn < v0.20.0
> from sklearn.covariance import GraphLassoCV as GraphicalLassoCV
>
> estimator = GraphicalLassoCV()
> estimator.fit(time_series)
> # Display the covariancec
> aas={}
> jsa=0
> for i in estimator.covariance_:
> r=list(a["Difumo_names"].values())[jsa]
> jsa=jsa+1
> a=dict()
>
>
> for x in range(64):
> g=list(a["Difumo_names"].values())[x]
>
> print(aas)
> t= nilearn.plotting.plot_img(estimator.covariance_, labels=list(a[
> "Difumo_names"].values()),
> figure=(9, 7), vmax=1, vmin=-1,
> title='Covariance')# The covariance can be found
> at estimator.covariance_
>
> # The covariance can be found at estimator.covariance_
> t2= nilearn.plotting.plot_matrix(estimator.covariance_, labels=list(a
> ["Difumo_names"].values()),
> figure=(9, 7), vmax=1, vmin=-1,
> title='Covariance')
>
>
>
> --
>
>
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Fwd: AUTO EDITOR DIDN'T WORK
I think it is time to ask this topic to go find some other place to talk to itself. I have seen NO reason to think any question about problems with Python has been asked. Not properly. It sounds like someone messed up an installation, perhaps of other programs like an editor and some unspecified thing does not work. I suggest they start over and be careful so that if some specific version of Python is needed, it gets installed in the right place and so on. We here cannot be expected to have much idea about programs that perhaps we do not use. Python can be used to build an editor, or parts it runs when needed, and it can be used to create or consume XML, or do things with audio formats. So can many other languages. If it was needed here and the setup was wrong or sabotaged, that is not something easily handled here. Or did I miss something? If so, I know others here also missed it too. If there is a more specific problem like some lines of actual python code not doing what was expected, please share that specifically with enough detail. -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Tuesday, June 13, 2023 10:18 PM To: python-list@python.org Subject: Re: Fwd: AUTO EDITOR DIDN'T WORK On 6/13/2023 9:43 PM, gene heskett via Python-list wrote: > On 6/13/23 19:10, Thomas Passin via Python-list wrote: >> On 6/13/2023 5:32 PM, Alan Gauld via Python-list wrote: >>> Okay thanks. Meanwhile, I am not tech savvy so I may not say much here. >>> I followed all the commands as given on the website to install auto >>> editor standing it on python but after rendering the XML file, I >>> couldn't open it with my Davinci Resolve 18. I uninstalled and >>> reinstalled about twice and still no success hence I uninstalled it. >> >> I don't understand when you talk about an "XML file". Auto-editor >> works on video files, or at least .mp4 files, which are not XML files. >> Davinci Resolve does have some ability to interoperate with other >> editors using XML in some way (according to Wikipedia, >> https://en.wikipedia.org/wiki/DaVinci_Resolve) but that's a different >> thing completely. >> >> I also don't know what you mean by "after rendering the XML file" >> since from what I can see auto-edit doesn't render anything. >> >> The simplest thing that auto-editor can do is to cut out long periods >> of dead space, e.g., from an mp4 file. Their documentation shows how >> to do it. If it were me, I would run the example command line on a >> sample mp4 file, then see what it looked like in Davinci. Is that >> what you did? It should be the same video but with some dead space >> removed. >> >> (Note that I'm speaking from a place of no experience with either of >> these software packages; just looking at what auto-edit claims to do). >> >> >> > auto-edit? Never heard of it. xml? I've written hundred of kilobytes of > it in plain old geany. I didn't know there was a special editor for xml. Oh, there are, there are - mostly intended for document authoring, I think. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Postgresql equivalent of Python's timeit?
Timing things that are fairly simple is hard enough to do repeatedly, but when it involves access to slower media and especially to network connections to servers, the number of things that can change are enormous. There are all kinds of caching at various levels depending on your hardware and resource contention with other programs running here and there as well as on various network-like structures and busses or just hard disks. Asking for anything to be repeated multiple times in a row as a general rule can make your results seem slower or faster depending on too many factors including what else is running on your machine. I am wondering if an approach to running something N times that may average things out a bit is to simply put in a pause. Have your program wait a few minutes between attempts and perhaps even do other things within your loop that make it likely some of the resources you want not to be in a queue have a chance to be flushed as other things take their place. Obviously, a machine or system with lots of resources may take more effort to use enough new data that replaces the old. Good luck. Getting reliable numbers is no easy feat as someone else may have trouble duplicating the results with a somewhat different setup. -Original Message- From: Python-list On Behalf Of Albert-Jan Roskam via Python-list Sent: Sunday, September 17, 2023 5:02 AM To: Peter J. Holzer Cc: python-list@python.org Subject: Re: Postgresql equivalent of Python's timeit? On Sep 15, 2023 19:45, "Peter J. Holzer via Python-list" wrote: On 2023-09-15 17:42:06 +0200, Albert-Jan Roskam via Python-list wrote: >This is more related to Postgresql than to Python, I hope this is ok. >I want to measure Postgres queries N times, much like Python timeit >(https://docs.python.org/3/library/timeit.html). I know about EXPLAIN >ANALYZE and psql \timing, but there's quite a bit of variation in the >times. Is there a timeit-like function in Postgresql? Why not simply call it n times from Python? (But be aware that calling the same query n times in a row is likely to be unrealistically fast because most of the data will already be in memory.) = Thanks, I'll give this a shot. Hopefully the caching is not an issue if I don't re-use the same database connection. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Where I do ask for a new feature
Bongo, Variables in most programming languages either have to be removed manually or allowed to drift outside a boundary when they disappear for scoping reasons and perhaps are garbage collected at some point. There are many ways to make transient variables that disappear at some time and do we need yet another? Yes, you can create one of those ways but what is the big deal with deleting a variable when no longer used? Examples might be the "finally" clause or the "with" statement or just putting the variable in a nested scope. -Original Message- From: Python-list On Behalf Of Bongo Ferno via Python-list Sent: Thursday, October 19, 2023 9:33 PM To: python-list@python.org Subject: Re: Where I do ask for a new feature > You can actually just do that with simple assignment! > > short_view = my_object.stuff.long_stuff.sub_object > print(short_view.some_method()) but then have to delete the variable manually del short_view -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Where I do ask for a new feature
I still see no great reason for a new feature here and the namespace issue has often been discussed. You can always opt to create your own namespace of some sort and make many of your variables within it and always refer to the variables explicitly so the only collisions that can happen are your own carelessness. I do note that reusing a variable like "i" is not uncommon and especially when it is merely used as a looping variable. Generally anything else using the same variable does not need to refer to the other use and is in another scope. May I politely ask if you can point to other languages that have the feature you want and what it looks like. How does it know when the variable can safely go away? Does it allow you to create your alias in something like a loop and re-assign it or is it more like a constant once set and so on? If you have a good use case with no other easy way to provide it, you still need to show it is more important than oodles of other feature requests before anyone would consider seriously doing it in some future release. I am wondering if your concept of an alias is more typographical than actual. I mean in languages like C, there was often a preprocessor that went through your code and made changes like: #DEFINE filename "/usr/me/dir/subdir/file.c" This could allow some shorter typing but would not have anything in the namespace on the compiler level which would just see the longer substitutions. Could Python include something like this by keeping a table of symbols and replacements or running a pre-processor first? Maybe. But as stated, it does not seem to be a NEED that some feel is important. Assigning a variable to hold a pointer of sorts does indeed add to the namespace but the resource involved is not a big deal. Python is an interpreted language that makes one pass but there are languages that make multiple passes through the code and allow things like defining a function after it has been called. That is not pythonic. -Original Message- From: Python-list On Behalf Of Roel Schroeven via Python-list Sent: Friday, October 20, 2023 3:55 AM To: python-list@python.org Subject: Re: Where I do ask for a new feature Op 20/10/2023 om 5:16 schreef Bongo Ferno via Python-list: > On Thursday, October 19, 2023 at 11:26:52 PM UTC-3, avi.e...@gmail.com wrote: > > > There are many ways to make transient variables that disappear at some time > > and do we need yet another? Yes, you can create one of those ways but what > > is the big deal with deleting a variable when no longer used? > > Assigning a variable to something can be anything else than a temporal alias. > A with statement makes clear that the alias is an alias and is local, and it > automatically clears the variable after the block code is used. > > Python clutters the variable space with vars that are needed only on certain > places, and an alias doesn't has a scope. > Convenient alias are short names, and short names are limited in quantity. If > the space is cluttered with short alias, it opens risks for wrong utilization. > > Its like writing a "for i" in a list comprehension and having to worry if "i" > was already used in another place.. As long as functions are kept reasonably short, which is a good idea anyway, I don't really see any of that as a problem. -- "Experience is that marvelous thing that enables you to recognize a mistake when you make it again." -- Franklin P. Jones -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
Whoa! The question cannot be about whether it is possible to prove any abstract program will be correct and especially not on real hardware that can fail in various ways or have unexpected race conditions or interacts with other places such as over the internet. It has been quite well proven (think Kurt Gödel) that any system as complex as just arithmetic can have propositions that can neither be proven as true or as false and could be either. So there will be logical setups, written perhaps into the form of programs, that cannot be proven to work right, or even just halt someday when done. The real question is way more detailed and complex. How does one create a complex program while taking care to minimize as well as you can the chances it is flawed under some conditions. There are walls of books written on such topics and they range from ways to write the software, perhaps in small modules that can be tested and then combined into larger subunits that can also be tested. There are compilers/interpreters/linters and sometimes ways of declaring your intentions to them, that can catch some kinds of possible errors, or force you to find another way to do things. You can hire teams of people to create test cases and try them or automate them. You can fill the code with all kinds of tests and conditionals even at run time that guarantee to handle any kinds of data/arguments handed to it and do something valid or fail with stated reasons. You can generate all kinds of logs to help establish the right things are happening or catch some errors. But all that gets you typically is fewer bugs and software that is very expensive to create and decades to produce and by that time, you have lost your market to others who settle for less. Consider an example of bit rot. I mean what if your CPU or hard disk has a location where you can write a byte and read it back multiple times and sometimes get the wrong result. To be really cautions, you might need your software to write something in multiple locations and when it reads it back in, check all of them and if most agree, ignore the one or two that don't while blocking that memory area off and moving your data elsewhere. Or consider a memory leak that happens rarely but if a program runs for years or decades, may end up causing an unanticipated error. You can only do so much. So once you have some idea what language you want to use and what development environment and so on, research what tools and methods are available and see what you can afford to do. But if you have also not chosen your target architecture and are being asked to GUARANTEE things from afar, that opens a whole new set of issues. I was on a project once where we had a sort of networked system of machines exchanging things like email and we tested it. A while later, we decided to buy and add more machines of a new kind and had a heterogeneous network. Unfortunately, some tests had not been done with messages of a size that turned out to not be allowed on one set of machines as too big but were allowed on the other that had a higher limit. We caught the error in the field when a message of that size was sent and then got caught in junkmail later as the receiving or intermediate machine was not expecting to be the one dealing with it. We then lowered the maximum allowed size on all architectures to the capacity of the weakest one. This reminds me a bit of questions about languages that are free and come pretty much without guarantees or support. Is it safe to use them? I mean could they be harboring back doors or spying on you? Will you get a guarantee they won't switch to a version 3.0 that is incompatible with some features your software used? The short answer is there are no guarantees albeit maybe you can purchase some assurances and services from some third party who might be able to help you with the open-source software. Unless your project accepts the realities, why start? -Original Message- From: Python-list On Behalf Of o1bigtenor via Python-list Sent: Tuesday, October 24, 2023 7:15 PM To: Thomas Passin Cc: python-list@python.org Subject: Re: Question(s) On Tue, Oct 24, 2023 at 6:09 PM Thomas Passin via Python-list wrote: > snip > > By now you have read many responses that basically say that you cannot > prove that a given program has no errors, even apart from the hardware > question. Even if it could be done, the kind of specification that you > would need would in itself be difficult to create, read, and understand, > and would be subject to bugs itself. > > Something less ambitious than a full proof of correctness of an > arbitrary program can sometimes be achieved. The programming team for > the Apollo moon mission developed a system which, if you would write > your requirements in a certain way, could generate correct C code for them. > > You won't be doing that. > > Here I want to point out something else. You
RE: Question(s)
Agreed, Chris. There are many methods way better than the sort of RAID architecture I supplied as AN EXAMPLE easy to understand. But even so, if a hard disk or memory chip is fried or a nuclear bomb takes out all servers in or near a city, you would need some truly crazy architectures with info not only distributed across the globe but perhaps also to various space satellites or servers kept ever further out and eventually in hyperspace or within a black hole (might be write-only, alas). The point many of us keep saying is there can not easily or even with great difficult, any perfect scheme that guarantees nothing will go wrong with the software, hardware, the people using it and so on. And in the real world, as compared to the reel world, many programs cannot remain static. Picture a program that includes many tax laws and implementations that has to be changed at least yearly as laws change. Some near-perfect code now has to either be patched with lots of errors possible, or redesigned from scratch and if it takes long enough, will come out after yet more changes and thus be wrong. A decent question you can ask is if the language this forum is supposed to be on, is better in some ways to provide the kind of Teflon-coated code he wants. Are there features better avoided? How do you make sure updates to modules you use and trust are managed as they may break your code. Stuff like that is not as abstract. In my view, one consideration can be that when people can examine your source code in the original language, that can open up ways others might find ways to break it, more so than a compiled program that you only can read in a more opaque way. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Tuesday, October 24, 2023 9:41 PM To: python-list@python.org Subject: Re: Question(s) On Wed, 25 Oct 2023 at 12:20, AVI GROSS via Python-list wrote: > Consider an example of bit rot. I mean what if your CPU or hard disk has a location where you can write a byte and read it back multiple times and sometimes get the wrong result. To be really cautions, you might need your software to write something in multiple locations and when it reads it back in, check all of them and if most agree, ignore the one or two that don't while blocking that memory area off and moving your data elsewhere. Or consider a memory leak that happens rarely but if a program runs for years or decades, may end up causing an unanticipated error. > True, but there are FAR more efficient ways to do error correction :) Hamming codes give you single-bit correction and two-bit detection at a cost of log N bits, which is incredibly cheap - even if you were to go for a granularity of 64 bytes (one cache line in a modern Intel CPU), you would need just 11 bits of Hamming code for every 512 bits of data and you can guarantee to fix any single-bit error in any cache line. The "if most agree, ignore the one or two that don't" design implies that you're writing to an absolute minimum of three places, and in order to be able to ignore two that disagree, you'd probably need five copies of everything - that is to say, to store 512 bits of data, you would need 2560 bits of storage. But with a Hamming code, you need just 523 bits to store 512 reliably. Here's a great run-down on how efficiently this can be done, and how easily. https://www.youtube.com/watch?v=X8jsijhllIA Side note: If we assume that random bit flips occur at a rate of one every X storage bits, having redundant copies of data will increase the chances of a failure happening. For example, using a naive and horrendously wasteful "store 256 copies of everything" strategy, you would be 256 times more likely to have a random bitflip, which is insane :) You would also be able to guarantee detection of up to 128 random bitflips. But as you can see, this puts a maximum on your storage ratio. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
I am replying to this which is heading into another topic: "(Tongue in cheek) Except doesn't one make more when software in hidden in an unreadable state? (That forces the user to go back to the original dev or group - - yes?)" Like some of us, I come from a time when much software was compiled. Sure, you could play interactively with the BASIC interpreter, but much else could be delivered to you as an executable and it was not easy to know what it did without trying it and certainly it was not easy to make changes to it and resell it as your own. Where does python fit in? On the one hand, it is completely visible as the software and modules you use tend to be stored on the machine being used in a readable state. If you come up with some nifty algorithm, it is there for anyone to see and copy or even alter if they have permissions. You can freely search a corpus of code to pick up interesting tidbits and that can be a plus but if your livelihood is based on selling your code or services, ... So do some people do things to make that harder? Can you deliver only files already converted to bytecode, for example? Could you have an interpreter that has special changes such as being able to take encrypted code and decrypt before using or perhaps have read privileges that normal users will not have? Obviously if your code is on a server that users can only access indirectly and in a controlled manner, this is not as much of an issue. I will skip the anecdotes, but point out how sometimes compiled code may have a whole bunch of other problems, including when a user can sneak in your office and modify the source code behind your back or when a virus can insert itself. So to return to the main point, not that I am selling anything, what do developers using Python do to try to make sure they get properly paid and others do not just use their work without permission? -Original Message- From: o1bigtenor Sent: Wednesday, October 25, 2023 6:59 AM To: avi.e.gr...@gmail.com Cc: Chris Angelico ; python-list@python.org Subject: Re: Question(s) On Tue, Oct 24, 2023 at 9:36 PM AVI GROSS via Python-list wrote: > > Agreed, Chris. There are many methods way better than the sort of RAID > architecture I supplied as AN EXAMPLE easy to understand. But even so, if a > hard disk or memory chip is fried or a nuclear bomb takes out all servers in > or near a city, you would need some truly crazy architectures with info not > only distributed across the globe but perhaps also to various space > satellites or servers kept ever further out and eventually in hyperspace or > within a black hole (might be write-only, alas). > > The point many of us keep saying is there can not easily or even with great > difficult, any perfect scheme that guarantees nothing will go wrong with the > software, hardware, the people using it and so on. And in the real world, as > compared to the reel world, many programs cannot remain static. Picture a > program that includes many tax laws and implementations that has to be > changed at least yearly as laws change. Some near-perfect code now has to > either be patched with lots of errors possible, or redesigned from scratch > and if it takes long enough, will come out after yet more changes and thus > be wrong. > > A decent question you can ask is if the language this forum is supposed to > be on, is better in some ways to provide the kind of Teflon-coated code he > wants. Are there features better avoided? How do you make sure updates to > modules you use and trust are managed as they may break your code. Stuff > like that is not as abstract. The above are very interesting questions - - - - anyone care to tackle one, or some? > > In my view, one consideration can be that when people can examine your > source code in the original language, that can open up ways others might > find ways to break it, more so than a compiled program that you only can > read in a more opaque way. > (Tongue in cheek) Except doesn't one make more when software in hidden in an unreadable state? (That forces the user to go back to the original dev or group - - yes?) TIA -- https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
Just want to add that even when you can prove that an algorithm works absolutely positively, it will often fail on our every finite computers. Consider families of algorithms that do hill climbing to find a minimum and maximum and are guaranteed to get ever closer to a solution given infinite time. In real life, using something like 64 bit or 128 bit floating point representations, you can end up with all kinds of rounding errors and inexactness on some inputs so it goes into a loop of constantly bouncing back and forth between the two sides and never gets any closer to the peak. It may work 99.99% of the time and then mysteriously lock up on some new data. Or consider an algorithm some use in places like Vegas that is absolutely 100% guaranteed to win. If you bet $1 and lose, simply double your bet as many times as needed and eventually, you should win. Of course, one little problem is that all you ever win is $1. And you might lose 50 times in a row and spend hours and risk ever larger amounts to just win that dollar. Sure, you could just start betting a larger amount like a million dollars and eventually win a million dollars but how long can anyone keep doubling before they have to stop and lose it all. After enough doublings the vet is for billions of dollars and soon thereafter, more than the worth of everything on the planet. The algorithm is mathematically sound but the result given other realities is not. A last analogy is the division by zero issue. If your algorithm deals with infinitesimally smaller numbers, it may simply be rounded or truncated to exactly zero. The next time the algorithm does a division, you get a serious error. So perhaps a PROOF that a real computer program will work would require quite a few constraints. Python already by default supports integers limited only in size by available memory. This can avoid some of the overflow problems when all you are allowed is 64 bits but it remains a constraint and a danger as even a fairly simple algorithm you can PROVE will work, will still fail if your program uses these large integers in ways that make multiple such large integers on machines not designed to extend their memory into whole farms of machines or generates numbers like Googolplex factorial divided by googolplex raised to the log(Googolplex ) power. Some problems like the above are manageable as in setting limits and simply returning failure without crashing. Many well-designed programs can be trusted to work well as long as certain assumptions are honored. But often it simply is not true and things can change. Python actually is a good choice for quite a bit and often will not fail where some other environment might but there are few guarantees and thus people often program defensively even in places they expect no problems. As an example, I have written programs that ran for DAYS and generated many millions of files as they chugged along in a simulation and then mysteriously died. I did not bother trying to find out why one program it called failed that rarely to produce a result. I simply wrapped the section that called the occasional offender in the equivalent try/catch for that language and when it happened, did something appropriate and continued. The few occasional errors were a bug in someone else's code that should have handled whatever weird data I threw at it gracefully but didn't, so I added my overhead so I could catch it. The rare event did not matter much given the millions that gave me the analysis I wanted. But the point is even if my code had been certified as guaranteed to be bug free, any time I stepped outside by calling code from anyone else in a library, or an external program, it is no longer guaranteed. We are now spending quite a bit of time educating someone who seems to have taken on a task without really being generally knowledgeable about much outside the core language and how much of the answer to making the code as reliable as it can be may lie somewhat outside the program as just seen by the interpreter. And unless this is a one-shot deal, in the real world, programs keep getting modified and new features ofteh added and just fixing one bug can break other parts so you would need to verify things over and over and then freeze. -Original Message- From: Python-list On Behalf Of Michael F. Stemper via Python-list Sent: Wednesday, October 25, 2023 9:34 AM To: python-list@python.org Subject: Re: Question(s) On 24/10/2023 18.15, o1bigtenor wrote: > What is interesting about this is the absolute certainty that it is impossible > to program so that that program is provably correct. Not entirely true. If I was to write a program to calculate Fibonacci numbers, or echo back user input, that program could be proven correct. But, there is a huge set of programs for which it is not possible to prove correctness. In fact, there is a huge (countably infinite) set of programs for
RE: Question(s)
I am not one for IDLE worship, Tenor. But if you have been getting a message here, it is that there are an amazing number of programs that support your use of python during the development phase and perhaps later. I actually often use an environment called RSTUDIO (now part of a new name of POSIT) because it has been expanded beyond supporting R and supports Python and a growing number of other languages or combos that combine word processing with inserts from multiple languages. I have not used it for other languages like C/C++ and Javascript but the point is that like some others, it is not specific to Python but provides support for it. And, somewhat amusingly, you can write programs that combine parts in R and in Python that inter-operate with each other. Since you are taking a fairly overwhelming challenge of trying to learn everything at once while also developing something you want to be perfect and with few if any flaws, it may make sense to start with more of an ASCII editor or something with a few features but a simple interface, and a bit later when some parts of your code are working and you have some experience, you can move the code up a few notches to tools that perform a lot more validation for you. Please consider that resources trying to teach you the language, besides often showing simpler scenarios, often are not going to tell you EVERYTHING else you may need or that is usable let alone teach you all available modules and so on. -Original Message- From: Python-list On Behalf Of o1bigtenor via Python-list Sent: Thursday, October 26, 2023 8:34 AM To: Michael Torrie Cc: python-list@python.org Subject: Re: Question(s) On Wed, Oct 25, 2023 at 10:19 AM Michael Torrie via Python-list wrote: > > On 10/25/23 05:51, o1bigtenor via Python-list wrote: > > Looks like I have another area to investigate. (grin!) > > Any suggestions? > > Seems to me you're trying to run before you have learned to walk. > > Slow down, go to the beginning and just learn python, write some code, > see if it runs. Go through the tutorial at > https://docs.python.org/3/tutorial/index.html Interesting - - - - ". . . see if it runs." - - - that's the issue! When the code is accessing sensors there isn't an easy way to check that the code is working until one has done the all of the physical construction. If I'm trying to control a pulsation system using square waves with distinct needs for timing etc I hadn't seen any way of 'stepping through the code' (phrase you use later). > > Your first and most basic tool is the python interpreter. It will tell > you when you try to run your code if you have syntax errors. It's true > that some errors the linters will catch won't show up as syntax errors, > but cross the bridge when you get to it. Once you have a basic grasp of > Python syntax, you can begin using some of the tools Python has for > organizing code: Functions and modules (eventually packages). > Eventually when your logic is placed neatly into functions, you can then > write other python programs that import those functions and feed > different parameters to them and test that the output is what you > expect. That is known as a test. > > Nothing wrong with geany as an editor. However, you might find the > Python Idle IDE useful (it usually installs with Python), as it lets you > work more interactively with your code, inspecting and interacting with > live python objects in memory. It also integrates debugging > functionality to let you step through your code one line at a time and > watch variables and how they change. I have been following this list for some time. Don't believe that I've ever seen anything where anyone was referred to 'Idle'. In reading other user group threads I have heard lots about java and its ide - - - don't remember, again, any re: an ide for python. Even in maker threads - - - say for arduino - - its 'use this cut and paste method of programming' with no mention of any kind of ide when it was microPython - - although being a subset of python it Idle may not work with it. > > When you encounter isses with your code (syntax or logical) that you > can't solve, you can come to the list, show your code and the full > output of the interpreter that shows the complete error message and back > trace and I think you'll get a lot of helpful responses. > -- That was the plan. My problem is that I'm needing to move quite quickly from 'hello, world' to something quite a bit more complex. Most of the instruction stuff I've run into assumes that one is programming only for the joy of learning to program where I've got things I want to do and - - - sadly - - - they're not sorta like the run of the mill stuff. Oh well - - - I am working on things! Thanks for the ideas and the assistance! Regards -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
Thomas, It looks like much of our discussion and attempts at help are not going to be that helpful to Tenor as we may be way off bass about what he wants to do and certainly RSTUDIO and quite a few other suggestions may not be available in his microcontroller. As I see it, some of his objective involves sampling a sensor in real time. I have not heard what he wants to do with the data gathered and this may be an example of where code needs to be running fast enough to keep up. Proving the code will work, especially if you add logging or print statements or run it in a monitored mode so you can follow what it is doing, presents special challenges. Now if he ever wants to read in a .CSV file and analyze the data and make graphs and so on, I might chime in. For now, I am dropping out. Avi -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Thursday, October 26, 2023 6:50 PM To: python-list@python.org Subject: Re: Question(s) On 10/26/2023 6:36 PM, AVI GROSS via Python-list wrote: > I am not one for IDLE worship, Tenor. But if you have been getting a message here, it is that there are an amazing number of programs that support your use of python during the development phase and perhaps later. I actually often use an environment called RSTUDIO (now part of a new name of POSIT) because it has been expanded beyond supporting R and supports Python and a growing number of other languages or combos that combine word processing with inserts from multiple languages. Excellent! I didn't know about this development. [snip] -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Yes, it would be nice if there was a syntax for sending a test message sort
of like an ACK that is not delivered to the recipient but merely results in
some status being sent back such as DELIVERABLE or NO SUCH USER or even
MAILBOX FULL.
An issue with the discussion that may be worth considering is that some
email addresses are not always valid or may not be valid yet but will be
activated later. If I plan on opening a business unit which DNS will later
support as specific.category.mycompany.com.au and we first want to write
some code and test it and roll everything out later, then a test for
u...@specific.category.mycompany.com.au could fail some tests now but may be
fine later. Or what if I turn my machine off on weekends and when it boots,
it sets up to be able to receive mail. Is the address only sometimes valid?
We cannot be sure what rules may change and for all we know, they will
select other UNICODE symbols to replace @ for use by countries not having an
@ on keyboards in the local language or support some syntax like {AT} to be
usable ...
I even wonder about a service along the lines of tinyurl where you register
a potentially long or complex or hard to type name and get a short readable
one instead that is just used to provide a re-direct or even changed
periodically to dynamically point to where you want them now, such for the
current day of the week. I can easily imagine them making a funny looking
email address such as user@TINYqwerty that may not pas your current test or
one that looks valid to you but maps into an invalid or even null address.
BTW, checking if an email is valid is much wider as a concept than whether
the email address looks like a possible address. A big check sometimes made
if if the headers in the message and various formatting issues look
reasonable or issues about attachments and even if it is passed by SPAM
detectors. This discussion is just about if an email address LOOKS possibly
valid or should not be accepted.
I note earlier iterations of email had addressed like
mach1!mach2!mach3!ihnp4!mach5!mach6!user or even mach1!mach2!user@mach3 and
I remember tools that analyzed what other machines various machines claimed
to have a direct connection to and tried to figure out a connection from
your source to destination, perhaps a shorter one or maybe a less expensive
one. Hence machines like ihnp4 and various universities that were densely
connected to others got lots of traffic. In that scenario, validity had
another meaning.
-Original Message-
From: Python-list On
Behalf Of D'Arcy Cain via Python-list
Sent: Wednesday, November 1, 2023 9:57 PM
To: python-list@python.org
Subject: Re: Checking if email is valid
On 2023-11-01 17:17, Chris Angelico via Python-list wrote:
> On Thu, 2 Nov 2023 at 08:09, Grant Edwards via Python-list
> wrote:
>> Make sure it has an '@' in it. Possibly require at least one '.'
>> after the '@'.
>
> No guarantee that there'll be a dot after the at. (Technically there's
> no guarantee of an at sign either, but email addresses without at
> signs are local-only, so in many contexts, you can assume there needs
> to be an at.)
druid!darcy - doesn't work any more but not because it is syntactically
incorrect.
Remember the good old days when we were able to test if an address
existed without sending? That was before the black hats discovered the
Internet.
--
D'Arcy J.M. Cain
System Administrator, Vex.Net
http://www.Vex.Net/ IM:da...@vex.net
VoIP: sip:da...@vex.net
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Yes, Chris, many things can be used for lesser purposes. Perhaps this could be like when people automate guessing passwords and one defense is to stop accepting after N bad guesses till some external method resets things. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Thursday, November 2, 2023 2:05 AM To: python-list@python.org Subject: Re: Checking if email is valid On Thu, 2 Nov 2023 at 15:20, AVI GROSS via Python-list wrote: > > Yes, it would be nice if there was a syntax for sending a test message sort > of like an ACK that is not delivered to the recipient but merely results in > some status being sent back such as DELIVERABLE or NO SUCH USER or even > MAILBOX FULL. > Yes, it would! Spammers would be able to use this syntax to figure out exactly which addresses actually have real people connected to it. It would save them so much trouble! Brilliant idea. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
I have never had a need to check email but in my reading over the years, I am aware of modules of multiple kinds you can use to do things like parsing dates, URL and email addresses and probably many other such things into some kind of object and then you can use aspects of the object to do interesting things and perhaps change some and then ask for the object to be placed back into some other format such as text. My guess is that a first test of an email address might be to see if a decent module of that kind fills out the object to your satisfaction. You can then perhaps test parts of the object, rather than everything at once, to see if it is obviously invalid. As an example, what does u...@alpha...com with what seems to be lots of meaningless periods, get parsed into? This may be another approach that reuses what may be well-designed and tested shared software. I wonder if there are also such modules that do quite a bit of what is asked which is to reject a large class of badly formed addresses. You could, of course, take what survives and run additional screens. In the end, this is a bit like junkmail where some light-AI algorithms go over a corpus of messages that humans have curated as junk or not junk and make some statistical decisions that are nonetheless often wrong. In that case, many humans nastily declare thinks as SPAM just because they do not want to get such messages. If you blasted out email alerts every time a child seems to have been kidnapped to everyone in the nation, how long before many such messages would become designated as SPAM? So is there any work where people have taken a decent collection of email addresses used in the past that turned out to be syntactically valid or not, and trained an algorithm to recognize most of them properly? That trained algorithm could be shared and incorporated into your programs either as the only method, or one you use in special cases. -Original Message- From: Python-list On Behalf Of Mike Dewhirst via Python-list Sent: Thursday, November 2, 2023 6:31 PM To: python-list@python.org Subject: Re: Checking if email is valid If i wanted an email verifier I would look at open source frameworks and see how they do it. Django comes to mind.--(Unsigned mail from my phone) Original message From: Michael Torrie via Python-list Date: 3/11/23 07:23 (GMT+10:00) To: python-list@python.org Subject: Re: Checking if email is valid On 11/2/23 00:42, Simon Connah via Python-list wrote:> Basically I'm writing unit tests and one of them passess in a string > with an invalid email address. I need to be able to check the string > to see if it is a valid email so that the unit test passess.If you truly have managed to code an RFC-compliant verifier, I commend you.> Valid as in conforms to the standard. Although having looked at the> standard that might be more difficult than originally planned.You'll have to read the relevant RFCs. Lots of corner cases! From whatI can see virtually no one on the internet gets it right, judging by thenumber of times I have valid email addresses flagged as not valid bypoor algorithms.-- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Chris, I don't mean anything specific in the abstract approach I outlined as a possible alternative to using one complex regular expression. My suggestion was that some of the work could be done by the module you used and THEN you may test various parts of the rest. For example, perhaps another module lets you test if the domain name is registered. I have not read the RFC and have not worked on email applications in decades and am not offering specific advice. I am merely suggesting the possible use of existing software modules that may provide a better approach in weeding out SOME bad addresses. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Friday, November 3, 2023 1:43 AM To: python-list@python.org Subject: Re: Checking if email is valid On Fri, 3 Nov 2023 at 12:21, AVI GROSS via Python-list wrote: > My guess is that a first test of an email address might be to see if a decent module of that kind fills out the object to your satisfaction. You can then perhaps test parts of the object, rather than everything at once, to see if it is obviously invalid. As an example, what does u...@alpha...com with what seems to be lots of meaningless periods, get parsed into? > What do you mean by "obviously invalid"? Have you read the RFC? ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Grant (and others), I am asking about the overall programming process of dealing with email addresses beyond checking the string for some validity. You mentioned requiring you type in your email twice as one example. I generally do a copy/paste to avoid typing or have my browser fill it in. Rarely the code is set to force me to actually type it in. And I note sites that force me to do too much typing of any kind or make me jump through hoops like having to get an email or text with a secondary number to type in or make me look at pictures and find the right ones and so on, encourage me to not use them much. There is a price for taking away convenience even if you see it as some form of security. Yes, there are tradeoffs. It really may be important to know what you want from your email addresses. If I sign YOU up for something like the Word of the day in a dozen languages by supplying your valid email address, then checking if it looks valid is less useful than sending an email to that address and asking the recipient to opt-in and verify they legitimately want it. If you want to ensure that your newsletter is still wanted, you may do something similar every year or so to everyone, or perhaps just those that have not had activity. If a mailbox starts rejecting messages, perhaps you send messages to their secondary contact info or just remove them. There are many such strategies and some may be way harder to implement than a simple and perhaps simplistic syntax check. I do wonder how much it sometimes matters when we see real-world scenarios where people who died a decade ago remain on voter registration rolls. If my mailing list has a few hundred bad emails on it, the costs of sending may be small albeit dealing with rejection messages may clog my logs. As for fake email addresses, there are many ways to play that game that are unlikely to be caught. Will they realize there is nobody at erew...@gmail.com? If you want to know if someone is going to sell your hello.th...@gmail.com address could you supply hell.othe...@gmail.com and then monitor mail that you will still receive as it seems google ignores periods in your email name? And, since others generally see the above as distinct, you can even use such a method to sign up for something multiple times. Complexity leaves room for loopholes. Still, obviously there are good reasons to do what you can to do some validation at many points along the way and especially when it may be critical. Asking someone to type in a new password twice when they cannot easily see what they are typing, is obviously useful as the consequence of losing it is high. Are getting the email addresses right as important? I know my wife registered a fairly common name of the jane@gmail.com variety that is now useless as it keeps receiving messages someone provided or typed in wrong that were supposed to go to janedoe@ or doe.jane@ or janedoe123@ or j.doe@ and so on. These include receipts, subscriptions to newsletters and much more. Some are inadvertent but the reality is she stopped using that email as it is now mostly full of SPAM as the others ... -Original Message- From: Python-list On Behalf Of Grant Edwards via Python-list Sent: Sunday, November 5, 2023 12:39 AM To: python-list@python.org Subject: Re: Checking if email is valid On 2023-11-04, Michael Torrie via Python-list wrote: > On 11/4/23 02:51, Simon Connah via Python-list wrote: > >> Wow. I'm half tempted to make a weird email address to see how many >> websites get it wrong. In my experience, they don't have to be very weird at all. >> Thank you for the link. > > Nearly all websites seem to reject simple correct email addresses > such as myemail+sometext@example.domain. I like to use this kind of > email address when I can to help me filter out the inevitable spam > that comes from companies selling off my address even after claiming > they won't. I've always suspected that's intentional. They refuse those sorts of e-mail addresses because they know that's what they are used for. If they allowed "plus suffixed" e-mail addresses, then all the crap they want to send to you would go into /dev/null where it belongs -- and we can't have that! > So I suspect that nearly all websites are going to reject other > kinds of weird email addresses you can create that are actually > correct. Definitely. Syntactic e-mail address "validation" is one of the most useless and widely broken things on the Interwebs. People who do anything other than require an '@' (and optionally make you enter the same @-containing string twice) are deluding themselves. -- Grant -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Just mildly noticing the topics discussed have wandered quite a bit away from Python, let alone even programming. Phone numbers are not what they used to be. They tend to be quite portable and in some ways can be chained so my house phone rings through to my cell phone but a text will not be forwarded. And, if I do not choose to read my texts, no amount of sending would enlighten me about your needs. I know people who get WhatsApp messages but not standard texts, for example. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Monday, November 6, 2023 6:20 PM To: python-list@python.org Subject: Re: Checking if email is valid On Tue, 7 Nov 2023 at 10:11, Greg Ewing via Python-list wrote: > > On 7/11/23 7:45 am, Mats Wichmann wrote: > > Continuing with the example, if you have a single phone number field, or > > let a mobile number be entered in a field marked for landline, you will > > probably assume you can text to that number. > > But if the site can detect that you've entered a mobile number into > the landline field or vice versa and reject it, then it can figure out > whether it can text to a given numner or not without you having > to tell it! > Oh yes, it totally can. Never mind that some mobile numbers are able to accept text messages but not calls, nor that some landline numbers can accept text messages as well as calls :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Python Golf
Discussions like this feel a bit silly after a while. How long something is
to type on a command line is not a major issue and brevity can lead to being
hard to remember too especially using obscure references.
Consider that the Perl version as shown below does not need to import
anything. If you had python import sys by default and perhaps even create a
briefer alias for sys.stdin, then this gets shorter:
py -c "import sys; print(sum(int(F.split()[1])for F in sys.stdin))" On
Behalf Of Jon Ribbens via Python-list
Sent: Tuesday, November 7, 2023 11:06 AM
To: python-list@python.org
Subject: Re: Python Golf
On 2023-11-07, Stefan Ram wrote:
> I read this in a shell newsgroup:
>
> perl -anE '$s += $F[1]; END {say $s}' in
>
> , so I wrote
>
> py -c "import sys; print(sum(int(F.split()[1])for F in sys.stdin))"
> to show that this is possible with Python too.
>
> But now people complain that it's longer than the Perl version.
>
> Do you see ways to make it shorter (beyond removing one space
> after the semicolon ";")?
It's a bit of an unfair competition given that, unlike Perl,
Python is not designed to be an 'awk' replacement.
Having said that, you could make it a bit shorter:
py -c "print(sum(int(F.split()[1])for F in open(0)))" https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Text messages have taken a nasty turn and especially now that so many people have unlimited messages per month in their plan. People overuse them to the point where I opt out of some things like my home town notifications as they bombard me with other things I am not interested in. A major offender now is various forms of security which insist on not letting you into a web site for your bank or other resources unless they first send you an email or text message or perhaps a voice call with random digits to use as verification. Try sitting somewhere with your phone muted as all the beeps get annoying. There are many reasons for preferences and I know many people find it harder to reply to texts on their phone than to emails on a PC with a full keyboard. But all of this is not really here or there. We are talking more about user interface design than about programming, let alone about Python. What strikes me as a useful direction is for people to suggest what resources and methods in the Python world are helpful. Examples would be modules that have been tested and used that do things well such as validating phone numbers or emails, perhaps flexibly so that if a validation fails, they prompt the user asking if they are sure it is correct and maybe offer to let them type it in again for verification. Other ideas as stated recently are routines that don't just ask for a number but specify the purpose, and perhaps messages about what circumstances would trigger a use of the number, such as if fraud is detected, and get you to opt in or refuse. Reusable libraries of sorts, or good documentation of examples, would perhaps help make User Interface design and customer satisfaction better and show Python as a good way to do some kinds of programs. In that light, I wonder if it makes sense to NOT insist people give you their email address at all, and make it optional so they do not need to provide you with something bogus just to go on. -Original Message- From: Python-list On Behalf Of D'Arcy Cain via Python-list Sent: Tuesday, November 7, 2023 11:24 AM To: python-list@python.org Subject: Re: Checking if email is valid On 2023-11-07 08:40, Grant Edwards via Python-list wrote: > If you, as a web developer, want the user to enter a text-message > capable phone number, then ASK FOR THAT! And you may as well ask if they even want you to send texts whether they can technically receive them or not. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:da...@vex.net VoIP: sip:da...@vybenetworks.com -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: xor operator
I was going to ask a dumb question. Has any other language you know of made something available that does what is being asked for and included it in the main program environment rather than an add-on? A secondary mention here has been whether short-circuiting functions like "any" and "all" have been augmented with something like "has_n" that evaluates arguments till it has found n or perhaps n+1 of what it wants then skips the rest. Does any language supply something like that? What would such a function return and does it have an "any" or an "all" side? It sounds like if I asked if a list of integers has at least n prime numbers in "any" mode, it should ignore any that are not primes till it finds n primes or fails and returns true or false. If in "all" mode, I assume it would have to be the first n items without a failure. Fine, but then someone may want to know WHERE you stopped or for you to return the sublist of the ones that made the match, or even return everything that was skipped so you can later process that. Consider a long list of jurors you process to place a dozen that qualify on a jury and then later you want to choose from among the rest for another jury. Human minds can come up with an amazing number of ideas including for "useful" functions or features but I find the vast majority would rarely be used as nobody remembers it is available and some fairly simple method using other functions can easily be cobbled together. -Original Message- From: Python-list On Behalf Of Grant Edwards via Python-list Sent: Monday, November 13, 2023 8:19 PM To: python-list@python.org Subject: Re: xor operator On 2023-11-14, Dom Grigonis via Python-list wrote: > >> Except the 'any' and 'all' builtins are _exactly_ the same as bitwise >> or and and applided to many bits. To do something "in line" with that >> using the 'xor' operator would return True for an odd number of True >> values and False for an even Number of True values. > > Fair point. > > Have you ever encountered the need for xor for many bits (the one > that I am NOT referring to)? Would be interested in what sort of > case it could be useful. Yes, it's used all the time in low-level communications protocols, where it's often implemented in hardware. But, it is also not at all unusual to implement it in software. It's also not that unusual for the "count-ones" part of the function you're asking for to be implemented in hardware by a CPU having an instruction that counts the number of 1 bits in a register. GCC has a low-level builtins called __builtin_popcount() and __builtin-popcountl() that counts the number of 1's in an unsigned (long) int. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: xor operator (DEPRECATED)
Dom, I hear you. As you say, writing your own extension in something like C++ may not appeal to you even if it is faster. I was wondering if using a generator or something similar in R might make sense. I mean what happens if you write a function that includes a "yield" or two and does a part of what you want. It maintains some internal state between invocations. So you can call it once to setup things then call it repeatedly to keep processing the next item. You stop calling it when you get a result you want, such as that it has seen what you want N times. Since the code stays in memory, it may effectively run faster than some other kinds of functions calls. It can keep things in internal storage such as not just how many N you want but how many it has seen. Your outer function can maintain a list of the items you want to XOR or generate a new one dynamically as needed. It can use functional programming techniques to create a new customized version of the iterator, such as with a value of N built in. You would then call the outer function and let it use the inner function till the result is available or until the data in the iterator runs out or perhaps other tweaks involving two way communication of sorts between the functions. I am NOT suggesting this approach is optimal or fast but merely wondering if something along these lines is worth trying that might speed things up even if not very fast. Such approaches can be even more effective if what you are working on need not all be instantiated up front but can be dynamically calculated or incrementally read from files. With care, you can make multiple instantiations that each iterate over their own sets of data without interference. Just a thought. In a sense, this can be a slightly decent substitute for the non-standard evaluation in R where you can arrange for lots of your data to not be interpreted till absolutely needed. -Original Message- From: Dom Grigonis Sent: Monday, November 13, 2023 10:12 PM To: avi.e.gr...@gmail.com Cc: Grant Edwards ; Python Subject: Re: xor operator (DEPRECATED) Fair point. However, I gave it a shot for the following reason: I couldn’t find a way to make such performant function. Using python builtin components still ends up several times slower than builtin `all`. Cython or numba or similar is not an option as they do not support `truth` values. Or if they do, it ends up slower than pure python variant. So all what is left is writing a proper extension. Which I would prefer not to do for 1 function. I thought maybe `xor`, as in logical XOR functionality in its vanilla case could be compelling. And after doing a bit of search I see that very very few languages have that and it seems for a good reason. Some that do: R, thats all I could find. Although some (if not many) went through the proposal phase. And yes, none of them have a function that I am proposing. So yes, you are right, not a good proposal. But there still seems to be the need for short-circuiting performant implementations in python space. The issue is that there are many variants of what might be needed while there is no efficient solution to sourcing predicates from python to lower level implementations. Someone mentioned that numpy experimented with such implementations in C, but they did not get anywhere with it. The best I could come up with is cached numba for numpy problems, which does perform very well and more than worth it if function is re-used. It even ends up faster than cython or cffi extensions, however can’t have many of those due to JIT and AOT is currently being deprecated (which wouldn’t solve anything anyway). However, as I mentioned earlier it does not apply to this case. So it’s either: a) Something very clever and flexible implemented that covers most of such needs and doesn’t require predicates. b) I welcome any thoughts on this. DG > On 14 Nov 2023, at 04:27, AVI GROSS via Python-list > wrote: > > I was going to ask a dumb question. Has any other language you know of made > something available that does what is being asked for and included it in the > main program environment rather than an add-on? > > A secondary mention here has been whether short-circuiting functions like > "any" and "all" have been augmented with something like "has_n" that > evaluates arguments till it has found n or perhaps n+1 of what it wants then > skips the rest. Does any language supply something like that? What would > such a function return and does it have an "any" or an "all" side? > > It sounds like if I asked if a list of integers has at least n prime numbers > in "any" mode, it should ignore any that are not primes till it finds n > primes or fails and returns true or false. If in "all" mode, I assume it > would have to be the first n items without
RE: Code improvement question
Many features like regular expressions can be mini languages that are designed
to be very powerful while also a tad cryptic to anyone not familiar.
But consider an alternative in some languages that may use some complex set of
nested function calls that each have names like match_white_space(2, 5) and
even if some are set up to be sort of readable, they can be a pain. Quite a few
problems can be solved nicely with a single regular expression or several in a
row with each one being fairly simple. Sometimes you can do parts using some of
the usual text manipulation functions built-in or in a module for either speed
or to simplify things so that the RE part is simpler and easier to follow.
And, as noted, Python allows ways to include comments in RE or ways to specify
extensions such as PERL-style and so on. Adding enough comments above or within
the code can help remind people or point to a reference and just explaining in
English (or the language of your choice that hopefully others later can
understand) can be helpful. You can spell out in whatever level of detail what
you expect your data to look like and what you want to match or extract and
then the RE may be easier to follow.
Of course the endless extensions added due to things like supporting UNICODE
have made some RE much harder to create or understand and sometimes the result
may not even be what you expected if something strange happens like the symbols
①❹⓸
The above might match digits and maybe be interpreted at some point as 12
dozen, which may even be appropriate but a bit of a surprise perhaps.
-Original Message-
From: Python-list On
Behalf Of Peter J. Holzer via Python-list
Sent: Friday, November 17, 2023 6:18 AM
To: python-list@python.org
Subject: Re: Code improvement question
On 2023-11-16 11:34:16 +1300, Rimu Atkinson via Python-list wrote:
> > > Why don't you use re.findall?
> > >
> > > re.findall(r'\b[0-9]{2,7}-[0-9]{2}-[0-9]{2}\b', txt)
> >
> > I think I can see what you did there but it won't make sense to me - or
> > whoever looks at the code - in future.
> >
> > That answers your specific question. However, I am in awe of people who
> > can just "do" regular expressions and I thank you very much for what
> > would have been a monumental effort had I tried it.
>
> I feel the same way about regex. If I can find a way to write something
> without regex I very much prefer to as regex usually adds complexity and
> hurts readability.
I find "straight" regexps very easy to write. There are only a handful
of constructs which are all very simple and you just string them
together. But then I've used regexps for 30+ years, so of course they
feel natural to me.
(Reading regexps may be a bit harder, exactly because they are to
simple: There is no abstraction, so a complicated pattern results in a
long regexp.)
There are some extensions to regexps which are conceptually harder, like
lookahead and lookbehind or nested contexts in Perl. I may need the
manual for those (especially because they are new(ish) and every
language uses a different syntax for them) or avoid them altogether.
Oh, and Python (just like Perl) allows you to embed whitespace and
comments into Regexps, which helps readability a lot if you have to
write long regexps.
> You might find https://regex101.com/ to be useful for testing your regex.
> You can enter in sample data and see if it matches.
>
> If I understood what your regex was trying to do I might be able to suggest
> some python to do the same thing. Is it just removing numbers from text?
Not "removing" them (as I understood it), but extracting them (i.e. find
and collect them).
> > > re.findall(r'\b[0-9]{2,7}-[0-9]{2}-[0-9]{2}\b', txt)
\b - a word boundary.
[0-9]{2,7} - 2 to 7 digits
- - a hyphen-minus
[0-9]{2} - exactly 2 digits
- - a hyphen-minus
[0-9]{2} - exactly 2 digits
\b - a word boundary.
Seems quite straightforward to me. I'll be impressed if you can write
that in Python in a way which is easier to read.
hp
--
_ | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| | | h...@hjp.at |-- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
--
https://mail.python.org/mailman/listinfo/python-list
RE: Newline (NuBe Question)
Grizz[l]y, I think the point is not about a sorted list or sorting in general It is about reasons why maintaining a data structure such as a list in a program can be useful beyond printing things once. There are many possible examples such as having a list of lists containing a record where the third item is a GPA for the student and writing a little list comprehension that selects a smaller list containing only students who are Magna Cum Laude or Summa Cum Laude. studs = [ ["Peter", 82, 3.53], ["Paul", 77, 2.83], ["Mary", 103, 3.82] ] magna = [stud for stud in studs if stud[2] >= 3.5 ] summa = [stud for stud in studs if stud[2] >= 3.75 ] print(studs, magna, summa, sep="\n") OUTPUT: >>> print(studs, magna, summa, sep="\n") [['Peter', 82, 3.53], ['Paul', 77, 2.83], ['Mary', 103, 3.82]] [['Peter', 82, 3.53], ['Mary', 103, 3.82]] [['Mary', 103, 3.82]] Of course, for serious work, some might suggest avoiding constructs like a list of lists and switch to using modules and data structures that are often more efficient to represent your data such as some form of matrix or data.frame. And, yes, you can sort something like the above by name or GPA or number of credits taken but the point was responding to why bother making a list just to print it. The answer is that many and even most programs do a bit more than that and a good choice of data structure facilitates ... -Original Message- From: Python-list On Behalf Of Grizzy Adams via Python-list Sent: Thursday, November 16, 2023 8:41 AM To: python-list@python.org Subject: Re: Newline (NuBe Question) Thursday, November 16, 2023 at 7:47, Thomas Passin via Python-list wrote: Re: Newline (NuBe Question) (at least in part) >I wrote that you don't need the "students" list, which is correct. But >there could be a use for a list. It would let you change the order in >which students appear in the printed output. Knowing how to do that is >a useful skill. But that should be left for a later lesson, not mixed >in here. I have a vague memory of seeing sorted list somewhere ;->) -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: RE: Newline (NuBe Question)
That is an entirely different discussion, Michael. I do not know what ideas Guido had ages ago and where he might stand now and I actually seriously disagree with the snippet you quoted below. Python was started long ago as a way to improve in some ways on what was there before. Some of the ideas were nice but also for some purposes, way too slow. If you regard the original versions of LISP, they too simplicity to an extreme and pretty much the main or even only data structure was a list. Functions like CAR and CDR accessed an element but a complex structure resulted in people creating functions with names like CAR and CADADADR to automate climbing a tree of sorts to get to the parts you want. It was a recipe for complexity and errors. My point was that although a list can do so many things in principle, it is not really optimized to do some things that can be way easier using add-ons or your own data structures like objects and he notes the collection library and deque as an example that he is not as much of a purist as you may think. My point was that if you have a fairly detailed and complex application that will manipulate lots of data, then instead of reading in a CSV with many columns and rows recorded into a list of lists, it may make sense to import numpy and pandas that come with all kinds of functionality built in so you do not need to re-invent everything. Just how easy is it using lists of lists to rearrange the order of columns of data, or add new columns containing calculations built from existing columns and so on? Of course, for many purposes, it is indeed overkill albeit once you learn a method, ... I think part of the design of Python, and this is just my guess, included going away from overly specific things done in earlier compiled languages and making it more abstract and inclusive. Arrays or vectors or other such names would normally require everything to be of the same data type and with a fixed length. The list data structure loosened this up quite a bit and also allowed lists within lists. That is great and for some purposes, not very efficient and especially not when your data actually is all of the same type or of fixed length. You can make matrix-like data structures of any depth using lists and it may be hard to traverse such as when you want to multiply two such 2-D matrices. Place the same data (all say floating point numbers) in a vector-like structure that also has stored info about the dimensions, and a simple mathematical calculation accesses any item such as may_tricks[5,42] in the same amount of time as an offset from the top. I have seen this phenomenon in many languages where a somewhat clean and sparse design gets added to, often by others, until some core features are used less often. An example would be R which does have lists nut they are just one form of vectors which are really more the core data idea. It also contains data.frames in the core which are implemented as a list of vectors and more recently a bit more. It was designed to do statistical tasks as one of the main objectives. Yet the graphics functions have been added to so there are by now quite a few independent ways to make graphics using different paradigms. Python also has something like that. And completely new paradigms such as piping data in a chain were added in packages and it became so popular that a version has been added to the core language. Now although the core language includes lots of the functionality you might see in numpy/pandas and you can do all kinds of things, some others kept creating new ways to do things including different data structures that either dealt with weaknesses found or were ore efficient and so on and an entire growing body of alternate ways to do things with lots more power and often more speed that I prefer. A collection of lots of these alternative packages has been assembled and I and others often simply start programs by including the "tidyverse" and the resulting programs might as well be written in a different language to anyone who only knows base R. But I do not see that as a bad thing albeit someone trying to get a part of a program from an AI-like service may need to specify or they may get code they cannot trivially read and evaluate but that works fine once they have loaded the packages. Guido is like many others who create or invent and do it in the way they are proud of. Why change it? But the reality is that first attempts are often done with lots of room for change and improvement. Now if you are teaching a course on Python basics, it may be a good idea to teach the basics and require students to only use in their homework what has already been taught. But if you get a job where the norm is to use modules like numpy, it makes sense to use the expanded language if it results in faster writing perhaps of faster code with fewer mistakes. -Original Message- From: Python-list On Behalf Of Michael F. Stemper via Python-list
RE: RE: Newline (NuBe Question)
Just FYI, I deliberately chose that abbreviation for a sort of irony as for some people college is about almost anything except learning and some people think they are studs and just party and ... And I am very tired of gender discussions. Lots of words now include two or even more genders. Women are often now "actors", not actresses. I see no reason women cannot be studs! But I learn from criticism. If I ever write a program like that and do not feel like typing, will this do? dents = [ ...] Or will that not include students who happen to be edentulous? -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Sunday, November 26, 2023 6:49 AM To: python-list@python.org Subject: Re: RE: Newline (NuBe Question) On Sun, 26 Nov 2023 at 21:08, Michael F. Stemper via Python-list wrote: > > On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote: > > Grizz[l]y, > > > > I think the point is not about a sorted list or sorting in general It is > > about reasons why maintaining a data structure such as a list in a program > > can be useful beyond printing things once. There are many possible examples > > such as having a list of lists containing a record where the third item is a > > GPA for the student and writing a little list comprehension that selects a > > smaller list containing only students who are Magna Cum Laude or Summa Cum > > Laude. > > > > studs = [ > >["Peter", 82, 3.53], > >["Paul", 77, 2.83], > >["Mary", 103, 3.82] > > ] > > I've seen Mary, and she didn't look like a "stud" to me. > That's what happens when you abbreviate "student" though :) Don't worry, there's far FAR worse around the place, and juvenile brains will always find things to snigger at, usually in mathematical libraries with "cumulative" functions. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Newline (NuBe Question)
Isn't it fascinating that a meaningless piece of code used to illustrate something can be analyzed as if it was full of malicious content? Yes, my choice of names was as expected. The numbers chosen had no special meaning other than choosing one number in each of three equivalence classes. But, if you want me to add subtle meaning for generations to examine as it it were a literary work, I offer this: Peter and Paul were studs who got Mary'd. Can we now go back to our regularly scheduled talking about aspects of a computer language? P.S. And just for history, Paul was really Noel Paul Stookey but Peter, Paul & Mary sounded more like new testament characters and I think Noel signifies a birth to Peter and Mary, sort of, which might have fit too unless it was a computer program where a name with an umlaut was once not common. Another interpretation is that Noel came from the Latin word for news. Be that as it may, and I have no interest in this topic, in the future I may use the ever popular names of Primus, Secundus and Tertius and get blamed for using Latin. -Original Message- From: Python-list On Behalf Of DL Neil via Python-list Sent: Sunday, November 26, 2023 4:58 PM To: python-list@python.org Subject: Re: Newline (NuBe Question) On 11/27/2023 12:48 AM, Chris Angelico via Python-list wrote: > On Sun, 26 Nov 2023 at 21:08, Michael F. Stemper via Python-list > wrote: >> >> On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote: >>> Grizz[l]y, >>> >>> I think the point is not about a sorted list or sorting in general It is >>> about reasons why maintaining a data structure such as a list in a program >>> can be useful beyond printing things once. There are many possible examples >>> such as having a list of lists containing a record where the third item is a >>> GPA for the student and writing a little list comprehension that selects a >>> smaller list containing only students who are Magna Cum Laude or Summa Cum >>> Laude. >>> >>> studs = [ >>> ["Peter", 82, 3.53], >>> ["Paul", 77, 2.83], >>> ["Mary", 103, 3.82] >>> ] >> >> I've seen Mary, and she didn't look like a "stud" to me. >> > > That's what happens when you abbreviate "student" though :) Don't > worry, there's far FAR worse around the place, and juvenile brains > will always find things to snigger at, usually in mathematical > libraries with "cumulative" functions. The OP used an abbreviation: "studs". Why? Too lazy to type the full word? Abbreviation has full-meaning in the (narrow) domain? Was wanting something funny, or to snigger over? Was the respondent sniggering? Perhaps he, like the OP, was also saving typing-time by making a joke, hoping that the OP would see the implicit-error in expecting others to understand that "studs" meant "students"? Actually, Peter, Paul, and Mary were a band (https://www.peterpaulandmary.com/), so "studs" is even less expressive when the data also tells a story... Working with "trainees", I avoid the word "student" even though some might see them as synonyms. In my mind, the abbreviation did not readily expand to the full word (mea culpa). Accordingly, would not pass Code Review! For the want of a few characters... (https://en.wikipedia.org/wiki/For_Want_of_a_Nail) -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Newline (NuBe Question)
Dave, Back on a hopefully more serious note, I want to make a bit of an analogy with what happens when you save data in a format like a .CSV file. Often you have a choice of including a header line giving names to the resulting columns, or not. If you read in the data to some structure, often to some variation I would loosely call a data.frame or perhaps something like a matrix, then without headers you have to specify what you want positionally or create your own names for columns to use. If names are already there, your program can manipulate things by using the names and if they are well chosen, with no studs among them, the resulting code can be quite readable. More importantly, if the data being read changes and includes additional columns or in a different order, your original program may run fine as long as the names of the columns you care about remain the same. Positional programs can be positioned to fail in quite subtle ways if the positions no longer apply. As I see it, many situations where some aspects are variable are not ideal for naming. A dictionary is an example that is useful when you have no idea how many items with unknown keys may be present. You can iterate over the names that are there, or use techniques that detect and deal with keys from your list that are not present. Not using names/keys here might involve a longer list with lots of empty slots to designate missing items, This clearly is not great when the data present is sparse or when the number of items is not known in advance or cannot be maintained in the right order. There are many other situations with assorted tradeoffs and to insist on using lists/tuples exclusively would be silly but at the same time, if you are using a list to hold the real and imaginary parts of a complex number, or the X/Y[/Z] coordinates of a point where the order is almost universally accepted, then maybe it is not worth using a data structure more complex or derived as the use may be obvious. I do recall odd methods sometimes used way back when I programmed in C/C++ or similar languages when some method was used to declare small constants like: #define FIRSTNAME 1 #define LASTNAME 2 Or concepts like "const GPA = 3" And so on, so code asking for student_record[LASTNAME] would be a tad more readable and if the order of entries somehow were different, just redefine the constant. In some sense, some of the data structures we are discussing, under the hood, actually may do something very similar as they remap the name to a small integer offset. Others may do much more or be slower but often add value in other ways. A full-blown class may not just encapsulate the names of components of an object but verify the validity of the contents or do logging or any number of other things. Using a list or tuple does nothing else. So if you need nothing else, they are often suitable and sometimes even preferable. -Original Message- From: Python-list On Behalf Of DL Neil via Python-list Sent: Sunday, November 26, 2023 5:19 PM To: python-list@python.org Subject: Re: Newline (NuBe Question) On 11/27/2023 10:04 AM, Peter J. Holzer via Python-list wrote: > On 2023-11-25 08:32:24 -0600, Michael F. Stemper via Python-list wrote: >> On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote: >>> Of course, for serious work, some might suggest avoiding constructs like a >>> list of lists and switch to using modules and data structures [...] >> >> Those who would recommend that approach do not appear to include Mr. >> Rossum, who said: >>Avoid overengineering data structures. >^^^ > > The key point here is *over*engineering. Don't make things more > complicated than they need to be. But also don't make them simpler than > necessary. > >>Tuples are better than objects (try namedtuple too though). > > If Guido thought that tuples would always be better than objects, then > Python wouldn't have objects. Why would he add such a complicated > feature to the language if he thought it was useless? > > The (unspoken?) context here is "if tuples are sufficient, then ..." At recent PUG-meetings I've listened to a colleague asking questions and conducting research on Python data-structures*, eg lists-of-lists cf lists-of-tuples, etc, etc. The "etc, etc" goes on for some time! Respecting the effort, even as it becomes boringly-detailed, am encouraging him to publish his findings. * sadly, he is resistant to OOP and included only a cursory look at custom-objects, and early in the process. His 'new thinking' has been to look at in-core databases and the speed-ups SQL (or other) might offer... However, his motivation came from a particular application, and to create a naming-system so that he could distinguish a list-of-lists structure from some other tabular abstraction. The latter enables the code to change data-format to speed the next process, without the coder losing-track of the data-type/format. The trouble is, wh
RE: Newline (NuBe Question)
Dave, I gave an example, again, and make no deep claims so your comments may be valid, without any argument. I mentioned CSV and a related family such as TSV as they were a common and simple data format that has long been used. There are oodles of others and yes, these days many people can read directly from formats like some from EXCEL. But for data that can be shared to almost anyone using anything, something like Comma Separated Values is often used. And some programs that generate such data simply keep appending a line at a time to a file and do not have any header line. There are even some programs that may not tolerate a file with a header line, or comments or other optional things, and some where header lines you can create would cause problems such as using an extended character set or escaped characters. I have worked with these files in many languages and environments and my thought process here focused on recent work in R, albeit much applies everywhere. My point was really not about CSV but the convenience and advantages of data structures you can access by name when you want and sometimes also by position when you want. Too many errors can happen when humans doing programming are not able to concentrate. It is similar to arguments about file names. In the old UNIX days, and the same for other systems like VMS, a filename tended to have a format where relatively few characters were allowed and it might have two parts with the latter being an extension of up to 3 characters, or whatever. So file names like A321G12.dat were common and also next to it similar unpronounceable other file names. It was easy to confuse them and even people who worked with them regularly would forget what it might mean or use the wrong one. Well, if I load in a CSV in a language like R and there is no header line, as with some other data structures, it may make up a placeholder set of names like V1, V2 and so on. Yes, there are ways to specify the names as they are read in or afterward and they can be changed. But I have seen lots of CSV files offered with way too many columns and no names as well as documentation suggesting what names can be added if you wish. This may be a bit off topic, but I want to add a bit in this context about additional concepts regarding name. As mentioned, there is a whole set of add-ons people sometimes use and in R, I like the tidyverse family and it allows some fairly sophisticated things to be done using names. There are ways to specify you want a subset of a data.frame (sometimes a version called a tibble) and you can ask for say all columns starting with "xyz" or containing it or ending with it. That can be very helpful if say we wave columns containing the height and weight and other metrics of say people in three clinics and your column names embed the name of the clinic, or other such examples, and you want to select one grouping for processing. You cannot easily do that without external info is it is just positional. An extension of this is how compactly you can do fairly complex things such as asking to create lots of new columns using calculations. You can specify, as above, which sets of columns to do this too and that you want the results for each XYY in XYZ.mean and XYZ.std and so on. You can skip oodles of carefully crafted and nested loops because of the ability to manipulate using column names at a high and often abstract level. And, just FYI, many other structures such as lists in R also support names for components. It can be very useful. But the overall paradigm compared to Python has major differences and I see strengths and weaknesses and tradeoffs. Your dictionary example is one of them as numpy/pandas often make good use of them as part of dealing with similar data.frame type structures that are often simpler or easier to code with. There is lots of AI discussion these days and some of what you say is applicable in that additional info besides names might be useful in the storage format to make processing it more useful. That is available in formats related to XML where fairly arbitrary markup can be made available. Have to head out as this is already long enough. -Original Message- From: 'DL Neil' Sent: Monday, November 27, 2023 2:49 AM To: avi.e.gr...@gmail.com; python-list@python.org Subject: Re: Newline (NuBe Question) Avi, On 11/27/2023 4:15 PM, avi.e.gr...@gmail.com wrote: > Dave, > > Back on a hopefully more serious note, I want to make a bit of an analogy > with what happens when you save data in a format like a .CSV file. > > Often you have a choice of including a header line giving names to the > resulting columns, or not. > > If you read in the data to some structure, often to some variation I would > loosely call a data.frame or perhaps something like a matrix, then without > headers you have to specify what you want positionally or create your own > names for columns to use. I
RE: why sqrt is not a built-in function?
Ethan, if it is not obvious, then should we add the following functions just
in case?
cube_root()
fourth_root()
nth(root)
two_thirds_root()
e_th_root()
pi_th_root()
x_over_y_root()
And so on.
It is true that square roots are probably more commonly taken than many
others above (I have never taken the pi_th root but you never know) and you
can make a fourth root by taking the square root twice. There is nothing
wrong with creating a convenience function but it needs to stop somewhere.
The purpose of modules or packages and other add-ons is to supplement the
base language that is loaded whether you want it or not. Some have argued to
remove many current things from the base that THEY never use (or compiled
their own such version) to make it smaller and faster.
Given that, you can either import the sqrt function or make your own that
just turns around and uses one of the methods that is built-in. But if you
use all kinds of things regularly, consider importing them all at once by
having your own module with some name that imports all of them and importing
that and waiting as they all load.
-Original Message-
From: Python-list On
Behalf Of Ethan Furman
Sent: Thursday, January 14, 2021 2:36 PM
To: python-list@python.org
Subject: Re: why sqrt is not a built-in function?
On 1/14/21 11:06 AM, Eli the Bearded wrote:
> "There should be one-- and preferably only one --obvious way to do it."
>
> Meanwhile, Alan Gauld pointed out:
>
>AG> because pow() is a builtin function and
>AG> root = pow(x,0.5)
>AG> is the same as
>AG> root = math.sqrt(x)
>
> Plus the ** operation ("root = x ** 0.5"), that's now three ways.
Yes, but which of those is obvious?
--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: list() strange behaviour
I am wondering how hard it would be to let some generators be resettable? I mean if you have a generator with initial conditions that change as it progresses, could it cache away those initial conditions and upon some signal, simply reset them? Objects of many kinds can be set up with say a reinit() method. I am not saying Python needs such a language change for generators as in many programs you could just recreate a new instance of a generator. But there may be places where by the time the generator is used, the original is not known. Or, there are places where you want to lengthen something to match another by repeatedly copying the same sequence as many times as needed. If a generator finishes, you want it to restart with the same sequence until you stop asking. This is just a thought, not a request for such a feature. -Original Message- From: Python-list On Behalf Of ast Sent: Saturday, January 23, 2021 2:54 AM To: python-list@python.org Subject: Re: list() strange behaviour Le 20/12/2020 à 21:00, danilob a écrit : > > > b = ((x[0] for x in a)) > There is a useless pair of parenthesis b = (x[0] for x in a) b is a GENERATOR expression first list(b) calls next method on b repetedly until b is empty. So it provides the "content" of b second list(b) provides nothing since b is empty (there is no reset on generators) -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: New Python implementation
I may be the only one who does not deal well with a condescending attitude. I have to wonder what international standards body ever completes a task in finite time, only to find the real world has moved on. Having standards can be a great idea. When the standard does not properly describe any implementations either because some leave out things and others have partial or enhanced implementations, then it is just a goal. May I ask if the proposed product itself needs standardization? Since it claims to support many (or amusingly ANY) language fully, perhaps they can share their Ada or other version before they do Python, or are they working on all of them at once? Realistically, many languages have chosen various paths and a model that captures them all will have to be fairly complex and perhaps needlessly complex. Does it need multiple ways to deal with issues like scope and perhaps keep track of that if a program crosses several boundaries? Will it be able to handle something like an R program running a package that allows a parallel running of a Python program as they work jointly on the same or copied data structures? I have been writing those lately and in the future may incorporate additional languages to take advantage of the strengths and features of each while avoiding the weaknesses or missing aspects of another. Anyone who considers the task specified to be a small problem is either brilliant or perhaps not well informed. If they can do what they say well, great. But I have seen other such attempts such as finding a way to translate between word processor formats that try to deal with overlapping but different concepts and do imperfect translations. That may of course not be relevant here if what is produced is code that runs and yet follows the expected rules as if it was being interpreted. But a more pleasant attitude may make the same points, not that I am sure what those are and what is being asked. It sounds more like being told. -Original Message- From: Python-list On Behalf Of Mr Flibble Sent: Thursday, February 11, 2021 1:15 PM To: python-list@python.org Subject: Re: New Python implementation On 11/02/2021 18:06, Chris Angelico wrote: > On Fri, Feb 12, 2021 at 5:01 AM Mr Flibble > wrote: >> >> On 11/02/2021 16:31, Dan Stromberg wrote: >>> On Thu, Feb 11, 2021 at 4:35 AM Mr Flibble >>> >>> wrote: >>> Hi! I am starting work on creating a new Python implementation from scratch using "neos" my universal compiler that can compile any programming language. I envision this implementation to be significantly faster than the currently extant Python implementations (which isn't a stretch given how poorly they perform). >>> >>> I'd like to encourage you to give this a go. It's a huge task, but >>> it's needed. >> >> Actually it is a relatively small task due to the neos universal compiler's >> architectural design. If it was a large task I wouldn't be doing it. >> >>> >>> You may be interested in the approaches of Pypy, Cython, Shedskin >>> and Nuitka. >> >> I am not particularly interested in any of the existing implementations as >> they bear no relation to the design of my language agnostic universal >> compiler, runtime, VM and JIT; the only use they will have will be to >> disambiguate certain Python language constructs that I cannot disambiguate >> from documentation alone: this is a natural consequence of Python not being >> standardized; those steering the language need to grow a pair and get Python >> standardized preferably as an ISO Standard. >> > > You keep insulting Python and the Python devs. Put up or shut up - > show some actual code before you make too many boasts. > > Python DOES have a strong language specification. Its semantics are > documented. If you find places where the documentation is lacking, > point them out specifically, don't FUD your way through. For a language to transition from "toy" status it has to be formally standardized. It is unacceptable to define a language in terms of a particular implementation. A git repo of Source code and associated observable dynamic behaviour when that code is compiled and ran is a poor substitute for an official ISO Standard. /Flibble -- 😎 -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: mutating a deque whilst iterating over it
I agree both with the idea that it is not good to mutate things during iteration and that some things we want to do may seemingly require effectively something like a mutation. I want to consider what data structure might capture a normal activity like having a to-do-list for TODAY and another for further in the future. I might sit down in the morning and look at my day and the list of activities I was going to do. I might note new activities to add, some that are done or moot and can be removed and some that should be done earlier or deferred to later or even into the next day. This does not seem easy to do iteratively. Weirder, if I throw each item at the end, I end up with the same items in the same order. So creating a data structure (such as an object like a deque but with more specific functionality) might take some work. To begin, you might want an iteration protocol that locks it till done. Not necessarily because of mutation, though. Within the iteration you might have code asking for what I might consider delayed actions. You can ask for the current item to be deleted or moved to a data structure for the next day and it will not be done now but some reference might be stored inside the object such as a DELETE list and a MOVE list. You may also have lists with names like HIGH, MEDIUM and LOW or priorities from 1 to N. I don't mean python lists, just some kind of way of assigning some meaning to each item as you go. You may even want a way to break a task into multiple parts or declare a dependency between them such as one having to be done before the other. When you are done iterating, presumably the data structure will then reorder itself in a scenario where mutability is done harmlessly and a new list or deque or whatever is reconstituted and you can unlock. I do note that years ago I took a course in time management that ended up spending way too much time doing this kind of task on paper multiple time a day with lots of erasing or rewriting. The idea was to get more of the important things done. These days, I am so interrupt-driven that such a system is not useful albeit a nice automated way might be helpful as my day constantly mutates to become unrecognizable. There are project management tools along the lines I describe that try to manage timelines and dependencies across multiple groups and measure deliverables and note when something will cause delays in others and so on. Obviously, beyond the scope but my point is they do not necessarily operate in one pass over the list and are quite a bit more complex. -Original Message- From: Python-list On Behalf Of Dan Stromberg Sent: Saturday, February 13, 2021 2:13 PM To: duncan smith Cc: Python List Subject: Re: mutating a deque whilst iterating over it On Sat, Feb 13, 2021 at 10:25 AM duncan smith wrote: > On 12/02/2021 03:04, Terry Reedy wrote: > > On 2/11/2021 3:22 PM, duncan smith wrote: > > > >>It seems that I can mutate a deque while iterating over it > >> if I assign to an index, but not if I append to it. Is this the > >> intended behaviour? > > What I was really wondering was whether the behaviour is as it is > because of the implementation or because it's how deques should > ideally behave. i.e. In my implementation do I stick strictly to the > same API, or allow it to differ? In some places I'm jumping through > hoops to stick to the API, and (when it comes to iteration) I'm > jumping through different hoops for different types of container (e.g. > lists versus deques). BTW, the reason I am implementing these at all > is that my containers are on-disk. Cheers. > > collections.deque appears to take the approach of "allow everything > we can based on our implementation, and trust the client not to overuse features". In fact, in Python, "over abstraction" is often seen as a bad thing, mostly because it slows things down so much. If you want something more abstracted, you might have a look at: https://stromberg.dnsalias.org/~strombrg/linked-list/ https://pypi.org/project/linked_list_mod/ (Both URL's are about the same module) Note that linked_list_mod is quite a bit slower than a collections.deque. Deques use a clever-but-hidden trick to gain a lot of speed - it's really a linked list of built in lists, which gives better locality of reference that masks CPU caches very happy. linked_list_mod goes for abstraction and simplicity. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: New Python implementation
It is likely that people would understand better if spoken to properly so I have been listening and hopefully gaining a picture that I can share, and be corrected helpfully when wrong. My personal guess is that the project at hand is to do something very vaguely like what was done to the CURSES functionality ages ago in prehistory where lots of different terminals (and terminal emulators) used an assortment of control and escape sequences to make a text-based terminal do things like delete to the end of a line. The idea was to extract out all kinds of functionality used by pretty much all the terminals and save the info for say the hp2621 terminal either in a file with that name, or as entries in a file specifying many terminals or even in your environment. Then you made sure you had some variable like TERM set to hp2621 or whatever. If later you logged in on another kind of terminal, it would adjust dynamically for that without changing your program. When a program like vi(m) or emacs ran, it used the curses library which would dynamically reconfigure based on the terminal used and try to figure out the cheapest way (in terms of characters and I/O usually) to send extra escape sequences to update your screen as changes were made. Sometimes it would clear the screen and re-enter the new stuff and sometimes delete three lines then put in the replacement and so on. End of example. There are other examples like a format for documents that might take one of many ones like Word and make a common storage format that can easily be printed on any device that supports the features. The important aspect for me, is abstraction. My impression is the proposed project would abstract out the details of what any language can do and then examine one language after another (such as Ada or Python) and create some kind of description that can be stored, such as in a file. I have no idea what that might look like. I assume it would include what keywords there are or what variable names might look like or if some construct has an ending like "FI" or "endif" or "}" and it is only once this is completed, that the next phase can be used. It sounds like they would have what he considers a universal compiler that determines what language (perhaps even version) a file or group of files are using and then loads in info that metamorphizes it into an effective compiler for that language. It may not be that simple and I have no idea how it does that or how it outputs the result but it sounds like they have yet another language we might loosely compare to the byte stream used by JAVA and SCALA and some others that is half-digested and can be run by the JVM, or the Python version of something along those lines. In any case, I get the impression that this output will then look about the same no matter what language it came from. It will not require specific run-time environments but one overarching runtime that can run anything, again, using whatever abstraction or rules they come up with. And, in some cases, you may choose to go a step further and take this portable file format and compile it further down to an executable that runs on a specific machine and so on. I shudder at how well it will output error messages! Assuming this is feasible and well done, it might open quite a few doors in terms of designing both mini-languages and variants on existing ones and brand new ones with new ideas. You would have to find a way to describe your language as described above and as long as it is consistent in some ways, no need to ever build your own interpreter or compiler. It might be a bit like designing a new terminal (or just emulator) that has features you want. Mind you, some of the new features might require changes in the "Neo" something/data before it can be handled, but once added, any other language, within reason, might be able to add a similar feature and it should work. I hope I got this at least partially right and it is more informative that repeatedly telling people things like "Nope" as if this is a quiz and not an informative discussion. -Original Message- From: Python-list On Behalf Of Mr Flibble Sent: Saturday, February 13, 2021 7:07 PM To: python-list@python.org Subject: Re: New Python implementation On 13/02/2021 18:11, Alan Gauld wrote: > On 13/02/2021 16:09, Mr Flibble wrote: >> On 13/02/2021 00:01, Alan Gauld wrote: >>> I'm assuming it's a new executable interpreter that can run any >>> valid python code. Is that correct? >> >> It is a universal *compiler* so it compiles the python code to byte >> code and then optionally to machine code via a JIT which is then executed. > > OK, sorry for being dense, but just to be absolutely clear. > > You are going to create a Python compiler that will take existing > Python code and output a byte code file. (I assume the byte code is > not standard Python byte code?) And I assume the execution environment
Efficiency debates
I think we have discussed this a few times. There are tradeoffs in computer science and obviously a compiled language with some optimization using low-level data structures does much better at solving simple problems. Interpreted languages often have serious overhead to start with and allow all kinds of flexibility that comes with more costs such as functions that handle arbitrary types of arguments. When languages keep adding not only features but accommodate many programming styles such as object-oriented and functional or multiple ways to do the same thing, there is often more bloat. In python, there are almost always many ways to do anything! So, I find it easier to develop in such languages BUT if the program is to be run regularly and uses lots of resources and especially if it does not need many of the dynamic features, there are ways to try to tune it within Python or use a faster attached library of functions in languages like C for some parts or leaving Python entirely. If a program does not actually need much of what Python offers, fine. Use something else. I know in another language called R, that more and more parts are rewritten in a dialect of C/C++ so some things run quite fast if they call such functions. Users learn to use vectorized methods instead of loops to gain speed. I have recently seen discussions that some tried to solve using nontrivial regular expressions when simpler and faster methods using normal string functions would do. Unless that application also needed much more, it could easily be done in C or C++ or if needed in a very localized assembler. Python may be closer to one-size fits all but sometimes more than is needed. But it is not the only example of where people choose a sledge-hammer when a thumbtack will do. But as humans, we may do better working in ways that work well with our normal thinking process or one we can learn and feels natural. Every language I know has problems that look easy to solve using features in it and others that are painful to contemplate. Lately, I have done work combining Python parts with R parts to use the powerful features I already know how to use well in each. The new R STUDIO supports Python and quite a few other languages now as well as notebooks and document preparation and more. But I make no claim it is fast. It is very convenient and sometimes that is worth more especially for a one-off project. Those who want speed, go for it. If you can find libraries of functions or class prototypes that are just what you need, great. But if a language like Python has even more resources available in lots of modules or packages as well as more flexible data types, then maybe a hundred lines of that will be easier to write than thousands of lines elsewhere. How convenient is it to work with data.frame type objects in C/C++ as compared to using numpy and pandas? For many of us, we work with machines with ever more power and most of the time it just sits there idling. Now if you have something on servers that is running many simultaneous copies all the time to process complex requests from users, then efficiency should matter. If you pay for it or worry about how much electricity it uses, fine. This forum does not care if you prefer C/C++ and is meant to discuss Python for people interested in Python. Repeated attacks make people ignore the person and not take anything they say as meaningful. I tried sharing my understanding of a recent proposal and remain unconvinced it is trivial and certainly doubt that it is doable as it is too grandiose and suggests any imaginable computer language will trivially be handled. I am sure it would have fun dealing with a programming language in Hebrew with most text going from right to left but numbers the other way, and with added features built-in that nobody is using yet. Oh, I have no intention of doing so, but I suggest a more doable goal would be to be able to handle five procedural languages with no objects or functional programming before making bold claims. If that works, expand. A more efficient use of my time is to tell my mailer which messages to toss. -- https://mail.python.org/mailman/listinfo/python-list
RE: New Python implementation
Grant, Haven't thought about Prolog in a LOOONG time but it had some wild twists on how to specify a problem that might not be trivial to integrate with other languages as our now seemingly censored person with much delusion of grandeur suggests. It is a language that does not specify what to do but more what rules an answer must abide by. Perhaps he should go to work for the Star Trek Federation and improving their Universal Translator that can learn any alien language in about two sentences. I am sure he will tell us it would be trivial for him or her along with other Fibbles he/she tells. Am I the only one who found it amusing, back to Python, that a recent attack on Python was about a fairly trivial problem to solve in most languages, let alone Python. Toy language does not normally apply to a fairly mature language, regularly extended to do many things in many ways, unless anything not fully standardized is a toy. I consider many such to become toys as they end up near the end of their lives and hard to extend further. I think the question was as simple as how to find the first period in a string (assuming it exists?) and replace it with an @ and not subsequent ones. Not very challenging even using very basic commands in a computer language. Computing 101? Can you name any language where that is hard to do from scratch? Sure, many will provide a ready-made function that does it in one line of code (or a partial line) but most languages let you use something like a loop that lets you look at one letter of a "string" variable at a time and perhaps copy it to a new one. A logical variable can be set so you conditionally replace just the first instance. Whether changed in place or in a copy, it seems trivial. So why snide comments that you need more than a toy language when this is precisely what is doable even in a toy language, but so commonly done that many better languages (and Python definitely is included) give you an assortment of built-in functionality to make it easy, and then dozens of other ways in modules you can load ... Now there are fairly complex things that a user cannot easily build from scratch or find a ready-made solution. There may well be say a PROLOG program that is simple and elegant and very hard to solve using Python, let alone a toy language. To me, all languages are toys, albeit for different age/experience groups. -Original Message- From: Python-list On Behalf Of Grant Edwards Sent: Monday, February 15, 2021 4:00 PM To: python-list@python.org Subject: Re: New Python implementation On 2021-02-15, Roel Schroeven wrote: > Is it your intention to not only compile procedural and > object-oriented languages, or also functional languages such as Haskell, Ocaml, Scheme? And Prolog! -- Grant -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: New Python implementation
Christian, Thanks for sharing. I took a look and he does have a few schemas for Ada and C from TWO YEARS ago. Nothing about the infinite number of other languages he plans on supporting, let alone Python. And what he has is likely not enough to do what he claims he can do easily and rapidly. What gets me is that many languages severely overload the use of some characters and tokens so they can mean many things based on the context and capturing that is much harder than declaring what is a keyword. Worse, some languages like PERL make so many things optional, that you can write very brief programs that keep implicitly assuming the previous part stored results in $_ and seem at first to be magical as the actual code does not seem connected to anything. I am not so sure there is a reasonable way to program ALL existing language his way & to have one ring-like compiler to rule them all, albeit you might be able to consolidate several somewhat similar languages. I also wonder how efficient such a universal compiler (maybe technically not a compiler) would be but if it was only used once, maybe not. But didn't he also say this thing would work in an interactive mode line by line? It reminds me of when I was studying concepts about various takes on a Turing Machine for my thesis. It has been proven that a Turing Machine can theoretically solve any problem you can solve with a non-quantum computer. I emphasize it is theoretical because we are talking about an infinite tape with squares containing symbols that are read by a head that moves over it either to the left or right based on the current state and so on. Fairly simple programs like copying a million instances of "1" in a row can take quite a while and really complex programs need near-infinite times or infinitesimal time per step. What is POSSIBLE is far from useful. I feel the same way about NEOS. Given a sufficiently advanced and convoluted schema you can write a sufficiently convoluted computer program that might halt with an answer -- meaning it will compile a valid program written in a language that is completely defined by that schema and perhaps additional info. We know it can be done because each language has existing compilers and/or interpreters that do so, without an explicit schema. But to have a program that can take any schema whatsoever and do what is needed is a bit much. Existing languages were simply not designed with this in mind and new/old languages might feel constrained about designing new features that will not be able to fit within an existing schema. In particular, how do you handle multiple versions of a language like Python 2.X and the incompatible Python 3.X and the eventual re-re-designed Python 4.Y? They all may be placed in files ending in .py! There is nothing wrong with thinking out of the box or making grand plans. But I have known people ranging from overly optimistic to manic who think nothing of making extraordinary claims and may actually believe it. I would not be shocked if a subset of what is being suggested for NEOS might be done by a large team in a few decades but by then, we may all be using quantum computers and a whole new set of languages to take advantage of new paradigms like superposition ... -Original Message- From: Python-list On Behalf Of Christian Gollwitzer Sent: Tuesday, February 16, 2021 2:25 AM To: python-list@python.org Subject: Re: New Python implementation Am 15.02.21 um 21:37 schrieb Roel Schroeven: > > So your claim is that your compiler is able to, or will be able to, > compile any language just by specifying a small schema file. Great! > > Do you maybe have a proof-of-concept? A simple language with a simple > schema file to test the basic workings of your compiler, Here is the git repo: https://github.com/i42output/neos under languages/ you'll find different schema files. Christian -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
School Python
I wonder if someone has come up with a sort of Python environment that lets kids play with more fundamental parts of the language that lets them get educated without the confusion. I mean a limited subset and with some additions/modifications. Someone mentioned how something like range(1,10) is confusing as it now not only does not produce a list on 1 to 9 or 1 to 10 but produces an object that only can be viewed well by doing more advanced things like iterating it. So why not have a module called something like EDUCATION which is loaded into the school version automagically and implements what you want. Make a function called SchoolTimeRANGE strange(here, there) (or whatever) that returns something like list( range(here, there)) so the student never sees a range object or yet knows there are objects to object to. If you want it to include the last item, include "there". Want it to start at 0 or 1, do that. As I see it, really limited languages make you work hard. Some of the LISP variants I used ages ago were so damn simple that you had programs largely consisting of combinations of CAR and CDR so dealing with some nested list data structure was painful enough that people had functions with name like CDDADAR to access some particular part. The same goes for various simple and limited languages that can do anything but won't keep a student in a room. Python is not limited and should not be but it should be taught progressively so basic computer concepts are learned before you learn arguably better ways. A beautiful thing about a list is it can be reused. Something with an iterator control can mysteriously be drained. So what is wrong with much of basic python, especially slightly extended? Feel free to tell them to use, strange() above for now as it lets them see what is happening step by step and mention that LATER they can (or must) switch to other functions normally used that may be more efficient or general. What I like about good parts of Python and that makes it good for teaching is that much can be written in almost normal English. Do you even need to use range(1,6) when you can ask the students to just type: numbers = [ 1, 2, 3, 4, 5 ] for index in numbers: ... Yes, later, introduce ways to loop from 1 to N by using better methods. List comprehensions? Why introduce them at all as most other languages do not have them and they are just syntactic sugar. If the goal is to introduce children to simple algorithms, keep them simple. Forget elegance or efficiency or being pythonic. But past a certain point, when they are ready, sure, add more. At some point it is helpful to learn that instead of keeping multiple arrays or variables in parallel, you could capture them in one object and tell the object what to do with itself and so on. Maybe later, show some tricks with functional programming. But don't START with elegant recursion methods. I see no major barrier to teaching using python for children who can handle indentation 😉 albeit they still have to learn to balance parentheses and quotes (sometimes) and brackets and braces ... What I think is good is that they can practice in front of an interpreter and get immediate results, not like I did with index cards that were handed in and returned a day later, or having to wait a long time for a compile to complete, with errors. -- https://mail.python.org/mailman/listinfo/python-list
RE: New Python implementation
Dennis made the interesting comment "... Python has too much built in ..." I understand his point. At the same time, I wonder what most people using computers today, or in the future, need. Given serious amounts of computer power, what many people may want is higher-level ways to get things done without worrying how they are done. Python, like many such languages, has a very basic core that is augmented by what could be libraries, packages, modules and so on but have been chosen to be built-in to one distribution or another. Some people and organizations use some things so commonly, they want python to start up automatically loading say numpy and pandas and other such things. They may not care if the programmer knows how to make a linked list or binary tree data structure. Many such details are encapsulated within objects built and tested already. I have seen subjects taught at various levels and python might qualify too. I took a Physics course that required Calculus and used it to derive formulas and other things. Others took one that sort of threw equations at you without much explanation. I have even seen a course called Physics for Poets. But at least the poets taking it knew science existed and perhaps to take it seriously when others who had studied it better shared their results, even if they have no interest in the methods. We routinely have people learn how to use Word Processors or Spreadsheets with no clue on how to build such a thing or anything about the Object models used within or even knowing there is such a thing. Do they need to know how to use the embedded methods to extend things with Visual Basic or Javascript? They like getting closer to a WYSIWYG interface that handles most of their usual needs and perhaps farm out harder things to experts when needed. So what if you teach some aspects of python that are needed all over, like how to make things conditional or in a loop, but not how to make every possible data structure. What if you deliberately do not teach functional aspects of the language at first or ever? For some people that is enough to enable them to then use additional instructions on how to find prepared and tested chunks to use, even if not always optimally or efficiently. For those who really want to be programmers, that still may actually be enough if their task is to work using the tools we have developed, not in making new tools from scratch. Quite a bit of programming today consists of reading manual pages and cobbling together chunks that together do the job. But obviously I would choose the classes where I understood more. Many here would. But where does it end? Do most of us know how compilers or interpreters get built or do we just work on top of existing implementations others make available? Can you build an operating system from Scratch or make microchips to run them on? If the goal is Computer USE literacy, I think python has more than enough if you include the modules that make hard things easy. Just a thought. Admittedly it is hard these days to give a homework assignment when the student can find a trivial way to get the result and not do the hard work. -Original Message- From: Python-list On Behalf Of Dennis Lee Bieber Sent: Thursday, February 18, 2021 12:45 AM To: python-list@python.org Subject: Re: New Python implementation On Tue, 16 Feb 2021 11:03:33 +, Alan Gauld via Python-list declaimed the following: >On 16/02/2021 07:35, Christian Gollwitzer wrote: >> Am 16.02.21 um 06:36 schrieb dn: >>> Pascal's value as a teaching language was that it embodied many >>> aspects of structured programming, and like Python, consisted of a >>> limited range of items which could be learned very quickly >> >> ROFL. Maybe that was true for Python when it was first invented. >> Today it is not "a few simple things". Even just the core language, > >Python v1 was a good teaching language. v2 complicated it a bit but it >was still usable. v3 is no longer a good teaching language (unless >maybe you are teaching CompSci at university.) > >In fact in v3 things are so complex that I seriously considered >dropping Python as the core language for my programming tutorial. >Until I started looking for an alternative, and there really isn't >anything much better. At least not that can also be used for real >projects once you've learned it. Depending upon the course intentions, I'd say Python is useful for teaching general programming and getting to usable real-world programs. For CompSci /theory/, OTOH, Python has too much built in, and would get in the way of having someone implement things like linked-lists, deques, hashed structures -- ie; the stuff that lies behind all those Python built-ins. Pascal, Modula-2 (or is it up to 3 now), or one of the other system programming languages: Edison from https://www.amazon.com/Programming-personal-computer-Brinch-Hansen/dp/013730 2673 Implement a hashed-head multiple-linked lis
RE: New Python implementation
Some of us here go way back and have stories to tell of what we did even before Python existed. I won't rehash my history here now except to say I did use PASCAL in graduate school and my first job before switching to C which was less annoying to use. What I am interested in, in this forum, is how Python grew and specifically what motivated adding new features. I see it as a bit more like a Camel created when a committee got together and decided to create a horse and one of them wanted it to also do well in a desert and so on. For teaching purposes, it can be useful to have a minimalist design and a way to catch lots of errors such as by strong typing. Some of that may also be useful in the real world. But in re-teaching someone now in another language, I keep running into the fact that when I use newer and more powerful features they feel overwhelmed as they vaguely remember the built-in way and the new way uses a piping metaphor they are not familiar with. I have had others who started with the new ways and don't even want to know how an earlier version of the language did it. In the real world, having to read, let alone maintain, code that others have had a hand in shaping and reshaping can be hard work if each person did things their own way. We have discussed the many ways you can format text in python and if a program uses them all, here and there, ... But for an individual programmer, it is great to use whichever method feels best for you, and especially if you came to python from another language that method was borrowed from or vice versa. Being a rich language has pro's and cons. LISP only had cons. -Original Message- From: Python-list On Behalf Of Alan Gauld via Python-list Sent: Friday, February 19, 2021 6:23 AM To: python-list@python.org Subject: Re: New Python implementation On 19/02/2021 03:51, Dennis Lee Bieber wrote: > They chose Pascal as being more modern, and something taught in > schools (yeah, like TurboPascal is going to be a good introduction to > writing software for real-time ground control of satellites). Funnily enough it was. Or at least for real-time telecomms control. We wrote all our real-time stuff on VAX and later PC using Pascal from the mid 80s through to early 1990s when we switched to C++. But TurboPascal was not much like Pascal, it had all the theoretical bits by-passed or removed. I still use Pascal in the shape of Delphi for building windows GUI apps today... But Delphi bears even less resemblance to Wirth's Pascal, in fact its quite similar to Python in many ways. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: New Python implementation
Benjamin, I wonder if you understood my intended meaning not about the plusses and minuses of using a language like LISP but that it is fundamentally build on using the CONS concept to make lists in a poetic way but has no PROSE. Not only does every language have what I meant by the usual meaning of pros and cons, but it depends on what other languages you are comparing it to, what kind of work you use the language for, is it for prototyping or final and efficient use, will someone else be extending or maintaining it, and which side of the bed you woke up on. I used to be a fan of brevity as in whatever make me type less. But over time, many functions you call now have so many arguments, that I am now a fan of specifying the names of each argument as in function_name(arg1=value1, arg3=value3, ...) because it makes it much clearer what you want and prevents a certain class of errors. Nonetheless, I despise very long variable names, even when the editor allows for name completion. I thus like to place many commands on multiple lines to be read somewhat vertically and short lines. Others prefer the opposite. If a language hinders this style of multi-line, it is a plus or minus depending. (cons "A" (cons "v" (cons "I" nil))) -Original Message- From: Python-list On Behalf Of Benjamin Schollnick Sent: Friday, February 19, 2021 1:31 PM To: Michael F. Stemper Cc: python-list@python.org Subject: Re: New Python implementation >> that method was borrowed from or vice versa. Being a rich language >> has pro's and cons. LISP only had cons. Now, Now. That’s certainly not correct. LISP does have a few Pros. Namely Job security. You’ll have a hard time replacing a experienced and professional LISP programmer. - Benjamin -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Is there a way to subtract 3 from every digit of a number?
Wouldn't it be nice, Grant, if Homework was assigned with statements like: "Using only the features of the language covered up to chapter 3, meaning individual variables and lists of them and simple loops and only using the arithmetic built-in variable of +, -, % ... Solve this problem " But there is an actual silly model and application to this homework assignment. Consider the kind of lock shown in the video (or skip the video) that has three or more wheels of sorts containing digits 0-9 and you rotate each wheel to a setting read down or across like 359. https://www.youtube.com/watch?v=BMeqkUiui20&feature=emb_logo If you are lazy, you can put the lock on your locker and move each wheel the same number of units in one direction. It is now securely locked and might show 682 or 026 and if nobody touches it and perturbs the setting, you can come back and perturb it back three units the other way (or continue seven more) and open it. See? A Very practical (albeit impractical) example of how this might be fractionally useful! I wrote a solution to the problem the student asked for that I chose not to share here that is one line of code including an embedded list comprehension to do the loop. If a student of mine in a beginning class offered me that solution, I would be fairly certain it was NOT their work, though, nor what I wanted them to do. Now the translate method, albeit elegant, is again not likely to be the one wanted as they probably have no idea that functionality exists. Heck, in some languages, they may not yet know looping constructs exist and be asked to use something like a GOTO! LOL! And, somewhere out there is something that implements the commonly (at least in the past) rot13 semi-cryptography of rotating the alphabet for fun and profit. You probably can load such a module and find a function that can rotate a numeric string by 3 or -3 and use that for a trivial solution. None of the above should be considered as having done the darn assignment as requested. However, if a student gave me a decent solution and ADDED that some search and research suggested other advanced methods they might use on the job later, sure, maybe they get extra credit. -Original Message- From: Python-list On Behalf Of Grant Edwards Sent: Saturday, February 20, 2021 12:31 PM To: python-list@python.org Subject: Re: Is there a way to subtract 3 from every digit of a number? On 2021-02-20, MRAB wrote: > Have a look at the 'translate' method of the 'str' class. That's very clever, but being too clever on homework assignemnts doesn't always get a good grade. If they've just studied iteration, the modulus operator, and int/str conversions, then I'd avise using the "dumb" method. -- Grant -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Is there a way to subtract 3 from every digit of a number?
Mike,
You moved the goalpost.
Some of us here have been speculating you were asking what we call a
homework question here. The problem seemed to be the kind asked for that can
be done using fairly simple commands in python combined together.
Of course, some of the answers posted used ideas usually not taught early
and that had to be explained. The one you liked uses anonymous functions
handed to a function that repeatedly applies it to a list of items created
from taking a number apart after it is I string form and then recombines it.
My method is straightforward enough and similar to another posting because
it just follows the logic.
>>> start = 1234567890987654321
>>> int(''.join([str((int(dig)+3) % 10) for dig in str(start)]))
4567890123210987654
But you just moved the goalpost by talking about using a data.frame as that
(and I assume numpy and pandas) are not very basic Python.
So think about it. You have a column in a data.frame. How do you apply other
transformations to it? In one sense, a column in a data.frame is just one of
many kinds of ordered collects in Python, not much different than a list.
How do you apply a set of instructions to take in one collection and replace
it with another computed item by item?
Since you know about making your own functions, you know that a clean way of
doing many things is to encapsulate the gory details in a function and call
it the usual way to apply the transformation. Sure, you could use another
anonymous function and make it messier to read.
And, if you know of the capabilities of a data.frame structure, you use the
method that it already offers and hand it your function in one of multiple
ways.
Can you satisfy our curiosity about what this functionality is for, now that
it seems you may have a bigger purpose? Or, is it HW for a more advanced
class?
-Original Message-
From: Python-list On
Behalf Of C W
Sent: Sunday, February 21, 2021 9:21 AM
To: Python
Subject: Re: Is there a way to subtract 3 from every digit of a number?
I do want to follow up, if I may. In Ming's example,
a = 2342
b = int("".join(map(lambda x: str((int(x)-3)%10) ,list(str(a)
What's the best approach to apply to a dataframe column, rather than just
one value? Here's my attempt using df[col_1''].apply(),
df['col_1'].apply(lambda:a int("".join(map(lambda x: str((int(x)-3)%10)
,list(str(a))
Thanks!
On Sun, Feb 21, 2021 at 9:12 AM C W wrote:
> Thanks so much everyone, I appreciate it!
>
> Ming, your solution is awesome. More importantly, very clear
> explanations on how and why. So, I appreciate that.
>
> Thanks again, cheers!
>
> Mike
>
> On Sun, Feb 21, 2021 at 12:08 AM <2qdxy4rzwzuui...@potatochowder.com>
> wrote:
>
>> On 2021-02-20 at 20:49:15 -0800,
>> Dan Stromberg wrote:
>>
>> > On Sat, Feb 20, 2021 at 7:13 PM Ming wrote:
>> >
>> > > I just wrote a very short code can fulfill your needs:
>> > >
>> > > a = 2342
>> > > b = int("".join(map(lambda x: str((int(x)-3)%10) ,list(str(a)
>> > >
>> > I tend to favor plenty of temporary variables with descriptive
>> > names,
>> but
>> > this is indeed short.
>> >
>> > Apart from that, you may find that using a generator expression is
>> shorter
>> > and clearer than map+lambda. It should allow to additionally
>> > eliminate
>> the
>> > list conversion.
>> >
>> > So in the terse form you've got there, it'd be more like:
>> > b = int(''.join(str((int(x) - 3) % 10) for x in str(a))
>> >
>> > But in real life, I'd try to use descriptive variable names for
>> > some of
>> the
>> > subexpressions in that. This makes reading and debugging simpler,
>> which is
>> > important because the maintenance phase of software is almost
>> > always
>> much
>> > longer and costly than the development phase. And although you
>> > could
>> do a
>> > generator expression for each of the different parts of (int(x) -
>> > 3) %
>> 10,
>> > I kinda like having a named function for just that piece.
>> >
>> > So maybe:
>> > def rot_3(character):
>> > """Convert to int, subtract 3 and mod 10."""
>> > digit = int(character)
>> > assert 0 <= digit <= 9
>> > return (digit - 3) % 10
>> >
>> >
>> > def descriptive_minus_three_caesar(input_number):
>> > """Convert to a -3 caesar cypher on an integer."""
>> > string_number = str(input_number)
>> > rotated_digits = (rot_3(character) for character in
string_number)
>> > output_string = ''.join(str(digit) for digit in rotated_digits)
>> > output_number = int(output_string)
>> > return output_number
>>
>> >>> descriptive_minus_three_caesar('38')
>> 5
>>
>> The problem is underspecified, and the examples are lacking, but
>> based on the phrase "each digit" and the examples that contain a 3,
>> I'd prefer to see "38" become "05."
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/p
RE: Is there a way to subtract 3 from every digit of a number?
Chris, I admit I must have missed that. I was trying to understand the overall request. I do most of my data.frame work in another language 😉 I will say that there are many problems that live at various levels. The original request could have been about how o covert a 1-digit integer which would have been simple enough. Instead it asked for any integer to be viewed in some way as a collection of single digits and manipulated individually. What we have now in effect with a column of a data.frame is a collection of those collections. If next he askes to convert all columns of the data.frame, we get to a collection of collections of collections 😉 But realistically, some of those collections already have methods in place so no need for something like writing code nesting loops within loops within loops. I deleted all the messages and I am asking myself if anyone else provided advice or actual code that zoomed in one how to do it to a series. You clearly saw it. -Original Message- From: Python-list On Behalf Of Chris Angelico Sent: Sunday, February 21, 2021 9:41 AM Cc: Python Subject: Re: Is there a way to subtract 3 from every digit of a number? On Mon, Feb 22, 2021 at 1:39 AM Avi Gross via Python-list wrote: > But you just moved the goalpost by talking about using a data.frame as > that (and I assume numpy and pandas) are not very basic Python. Given that the original post mentioned a pd.Series, I don't know how far the goalposts actually moved :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Is there a way to subtract 3 from every digit of a number?
Ah, that is an interesting, Mike, but not an informative answer. My question is where the specific problem came from. Yes, someone used to R and coming to Python might work at adjusting to what is different and how to get things done. I do that all the time as one hobby is learning lots of languages and assessing them against each other. So care to share your solution in R which I would assume you could do easily? My quick and dirty attempt, of no interest to the python community, using the R form of integer that has a maximum, and the pipe operator that will not be in standard R for another iteration, is this: ## R code using pipes to convert an integer ## to another integer by subtracting 3 ## from each digit and wrapping around from ## 2 to 9 and so on, meaning modulo 10 ## Load libraries to be used library(dplyr) ## define function to return subtraction by N mod 10 rotdown <- function(dig, by=3) (dig -by) %% 10 start <- 123456789L ## Using pipes that send output between operators start %>% as.character %>% strsplit(split="") %>% unlist %>% as.integer %>% rotdown %>% as.character %>% paste(collapse="") %>% as.integer When run: > start %>% + as.character %>% + strsplit(split="") %>% + unlist %>% + as.integer %>% + rotdown %>% + as.character %>% + paste(collapse="") %>% + as.integer [1] 890123456 The above is not meant to be efficient and I could do better if I take more than a few minutes but is straightforward and uses the vectorized approach so no obvious loops are needed. -Original Message- From: Python-list On Behalf Of C W Sent: Sunday, February 21, 2021 9:48 AM To: Chris Angelico Cc: Python Subject: Re: Is there a way to subtract 3 from every digit of a number? Hey Avi, I am a long time R user now using Python. So, this is my attempt to master the language. The problem for me is that I often have an idea about how things are done in R, but not sure to what functions are available in Python. I hope that clears up some confusion. Cheer! On Sun, Feb 21, 2021 at 9:44 AM Chris Angelico wrote: > On Mon, Feb 22, 2021 at 1:39 AM Avi Gross via Python-list > wrote: > > But you just moved the goalpost by talking about using a data.frame > > as > that > > (and I assume numpy and pandas) are not very basic Python. > > Given that the original post mentioned a pd.Series, I don't know how > far the goalposts actually moved :) > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: name for a mutually inclusive relationship
Is there a more general idea here? How about asking for a control that internally manages N items and requires exactly M of them before the entry is accepted when you click? The case being discussed sort of wants N out of N, or nothing. Example, you order a family dinner from a Restaurant and are told that for a fixed price, you must order exactly 4 items from column A and another 5 from column B. Now linking the two is a bit much but if there are ten items in column A, the only valid choices might be to not order at all or pick exactly 4. Picking your first item would be easy and picking your second and third too. Once you have 4, the setup should sort of lock so you cannot add more. But if you unclick one, you should be able to select another. To get out of the setup you either stop at exactly 4 or cancel out. The problem with the N out of N case is that it is all or none. Unclicking anything in this case may not be enough and perhaps the code should clear all other items too. Clicking on any one, should mark all of them. So not really a simple subset of the cases. And what messages does a user get as they use the control? -Original Message- From: Python-list On Behalf Of Ethan Furman Sent: Wednesday, February 24, 2021 5:14 PM To: python-list@python.org Subject: Re: name for a mutually inclusive relationship On 2/24/21 1:54 PM, 2qdxy4rzwzuui...@potatochowder.com wrote: > Ethan Furman wrote: >> I didn't say it was a good example. ;-) Hopefully it gets the idea across. > > Ditto. ;-) > > IMO, the whole idea of "my program has two options, and the user has > to specify both or neither," isn't a question of whether or not the > argument parsing library supports it, but a question of whether or not > it's a good API. Like I said, at this moment I don't have a good example, only an awareness that such a thing could exist and I don't know the name for it (if it has one). So far I have seen that there are even fewer good use-cases than I might have guessed, and that no one seems to have a name for the general idea. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: name for a mutually inclusive relationship
YAGNI? True, Chris, I never have. And if I ever did, I might not even know someone has implemented similar functionality with 86 optional function arguments that fine-tune what happens in various cases and what error messages to supply, LOL! So, I would end up re-implementing it myself. But isn't this true for so much software we discuss here. The usual mantra is that 80% of features are used maybe 20% of the time and a few are used never except maybe while testing. Realistically, it adds to bloat if a fairly well-defined object is extended to handle every possible feature that is allowed, and then some. We recently discussed the "+=" feature for an object when much of the time, it is pretty much as useful as just using the longer way of adding something to yourself. But loading a longer object description the 99% of the time when it is not used, also costs a bit. My comment was thus mostly academic and suggested realistic scenarios some people use commonly enough that might be possibly to implement to make it easier to deal with such a structure and that arguably such could also handle this edge case. No reason to think this is an important thing to add, just a category that could be doable. -Original Message- From: Python-list On Behalf Of Chris Angelico Sent: Thursday, February 25, 2021 1:14 AM To: Python Subject: Re: name for a mutually inclusive relationship On Thu, Feb 25, 2021 at 4:06 PM Avi Gross via Python-list wrote: > > Is there a more general idea here? How about asking for a control that > internally manages N items and requires exactly M of them before the > entry is accepted when you click? The case being discussed sort of > wants N out of N, or nothing. > YAGNI. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: a + not b
As a guess, Rob, precedence rules for not may not bind as strongly as you think. 1 + (not 1) With parentheses, "not 1" is a subexpression that should be performed first and might return the value "False" 1 + False treats False in a numeric context as a zero so evaluates to 1. But 1 + not 1 Looks a bit ambiguous as "+ " expects something it considers a number or that can be converted to a number. Without parentheses, it may try to do 1 + not Which gets the same Syntax error. I looked and operator "not" is evaluated much later than "+" and just before "and" and "or" so you need parentheses to force the interpretation you may intend. Similarly, some used of "and" require parentheses as do other operators. -Original Message- From: Python-list On Behalf Of Rob Cliffe via Python-list Sent: Wednesday, March 3, 2021 10:19 PM To: Python Subject: a + not b I can't work out why 1 + - 1 1 + (not 1) are legal syntax, but 1 + not 1 isn't. Is there a good reason for this? Thanks Rob Cliffe -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: neonumeric - C++ arbitrary precision arithmetic library
Just to be clear, and luckily the person who posed such boasts is luckily gone and I AM NOT interested in having the discussion again here, I have a point to make. He did not suggest a magical compiler that could translate any language. Not exactly. I think he suggested that if all languages share a large set of paradigms related to what methods we have for computing, then there may be a way to capture and store some description of any new language that defines it well enough. He proposed a sort of universal program, indeed, that would read the description files and then actual programs and generate something that ultimately would be runnable. Of course, he basically ignored any little details along the way and even insisted it would be easy and the result would be much more efficient in every case. So, I suggest he is not as bright as he thinks. I, on the other hand, think it is a hard problem but in some abstract sense doable given infinite resources and time. Translation, not a worthwhile quest for NOW. There are accepted theorems that suggests a fairly simple Turing Machine can solve any problem computable on our current (non-quantum) generation of computers. The programs would be very far from efficient and in some cases would require an infinitely long tape and more states than our universe can accommodate. But in principle, it can do anything as stated above. So, any program in any language can theoretically be rewritten as a Turing Machine version. But that would not be done by a universal translator. I assume you could do it from say a compiled machine language version for some languages but it would require access to anything modern programs get from all kinds of libraries out there that extend the program and if it spawns other processes or has things run in parallel it gets more and more complex. But why do it at all? Sure, translating parts of some programs in one language to another can make sense. But why a goal to have a sort of Unified Programming Language Theory other than as an academic exercise? Having said that, I keep looking at how languages like Python and R grow and how they keep adding features including many borrowed or imitated from elsewhere and I conclude you can just put everything imaginable into Python and the rest become superfluous! Problem solved. -Original Message- From: Python-list On Behalf Of Bonita Montero Sent: Saturday, March 6, 2021 2:12 PM To: python-list@python.org Subject: Re: neonumeric - C++ arbitrary precision arithmetic library >> There is no projection. >> _You_ have megalomania, not me. >> And there's also no Dunning Kruger effect. >> You can't assess your capabilites, not me. > no u Someone who says that he is capable of writing a compiler that translates every language has megalomania. No one can do this. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Apriori Algorithm
"I want to make apriori algorithm from start. Anybody have any reference file?" Excellent question. Based on everything you shared, I think all I can offer is that whatever you do, do not make the aposteriori version. Or, you could consider asking a real question with enough detail that maybe someone understanding it will give you a hint in advance. -Original Message- From: Python-list On Behalf Of sarang shah Sent: Saturday, March 6, 2021 9:47 PM To: python-list@python.org Subject: Apriori Algorithm I want to make apriori algorithm from start. Anybody have any reference file? -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Apriori Algorithm
I apologize for my earlier snide remark as I was not then aware there was an algorithm called apriori based on the Latin term and wondered if someone was pulling someone's leg, in advance. Someone has posted a pointer to Python code that is supposed to do that. If that is suitable, then the serious task is to read what they say and use it. I note the data you show below is ambiguous and not in the format that code asks for. Are you saying each line I see of what looks like integers is a grouping you want to contrast to other such lines in the algorithm? I doubt it can be used as is but needs to become some kind of data structure such as a list of tuples or whatever the algorithm wants. -Original Message- From: Python-list On Behalf Of dn via Python-list Sent: Sunday, March 7, 2021 3:09 AM To: python-list@python.org Subject: Re: Apriori Algorithm On 07/03/2021 20.56, sarang shah wrote: > I have this dataset in a text file I need to make an apriori algorithm based on it. Please help. > > 25 52 164 240 274 328 368 448 538 561 630 687 730 775 825 834 > 39 120 124 205 401 581 704 814 825 834 > 35 249 674 712 733 759 854 950 > 39 422 449 704 825 857 895 937 954 964 > 15 229 262 283 294 352 381 708 738 766 853 883 966 978 > 26 104 143 320 569 620 798 > 7 185 214 350 529 658 682 782 809 849 883 947 970 979 > 227 390 > 71 192 208 272 279 280 300 333 496 529 530 597 618 674 675 720 855 914 932 > 183 193 217 256 276 277 374 474 483 496 512 529 626 653 706 878 939 > 161 175 177 424 490 571 597 623 766 795 853 910 960 > 125 130 327 698 699 839 > 392 461 569 801 862 > 27 78 104 177 733 775 781 845 900 921 938 > 101 147 229 350 411 461 572 579 657 675 778 803 842 903 > 71 208 217 266 279 290 458 478 523 614 766 853 888 944 969 > 43 70 176 204 227 334 369 480 513 703 708 835 874 895 > 25 52 278 730 > 151 432 504 830 890 Great! For what purpose - is this a 'homework assignment'? What code do you have so far? -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: neonumeric - C++ arbitrary precision arithmetic library
The precedence example used below made a strange assumption that the imaginary program would not be told up-front what computer language it was being asked to convert from. That is not the scenario being discussed as we have described. In any particular language, there usually is a well-known precedence order such as "*" coming before "+" unless you use something like parentheses to make your intent clear and over-ride it. But it makes me think of a related question. Assume you are writing code in a language like Python and the language has been analyzed and all kinds of info stored that describes it and that includes things like precedence rules for operators. Suppose it works fine and YOU want to break it. Could you? If your language supports constructs like creating what looks like new infix operators as an example, or overloading existing ones, could you write expressions this mystical product would interpret differently than the original compiler or interpreter? As an example, what does this simple statement mean: A + B I mean A can be an integer or floating point number or character string but can also be a complex number or a data.frame or just about any arbitrary object. So can B. And in many cases, the operation is not at all Abelian and B+A is something else. In many cases, the logic is to look at the right-hand or left-hand variable and see if it has a method defined to be called such as __ADD__ or __RADD__ or maybe inherited such a method from another class perhaps using some specific method with complicated multiple inheritance scenarios. Or the language may choose to convert either A or B or both to some other format and perform some kind of addition and maybe convert the result to something. Lots of work is in the Python interpreter to deal with this and some of it has to be flexible enough to deal with times when a class or object is changed as the program runs. Might it be possible to create edge cases that break a static set of description files? If not, how much more complex must the imaginary program get to handle these challenges and what might that do to the purported gains in efficiency claimed? I am currently studying a language called Kotlin. They, like some other languages, allow the creation of new infix operators and once I have defined "myplus" I can write code like: A myplus B What if it would also allow me to specify dynamically what the priority of "myplus" is? Clearly it cannot be a reserved keyword in some table loaded in the language in advance. R lets me do something similar like create "%myplus%" as an operator and even weirder things like rebinding "+" to mean something else that no longer has a particular priority that was pre-ordained. So if we had a contest on finding ways to break a sort of general purpose customizable any-language mangler, I wonder if it might be easy to mangle it at first. To get around that, you might have to not handle languages you declare as misbehaving, or make a more complex system that handles all known cases. But even then, can we design a new language for the express purpose of exposing some hidden flaw and make them deal with that? New languages keep popping up, some with no prior intent on breaking anything, but I think a universal translator may not be imminent. -Original Message- From: Python-list On Behalf Of Peter J. Holzer Sent: Sunday, March 7, 2021 2:43 PM To: python-list@python.org Subject: Re: neonumeric - C++ arbitrary precision arithmetic library On 2021-03-06 23:41:10 +0100, Mirko via Python-list wrote: > I even wonder why they have tried. Writing a universal > compiler/interpreter sounds logically impossible to me, Here's a > simple Python expression: > > >>> 3+3*5 > 18 > > And here's the same expression in (GNU) Smalltalk: > > st> 3+3*5 > 30 > > > How would such a universal compiler know which evaluation strategy to > follow, if not by writing a parser end evaluator for every supported > language? That depends on what you mean by "writing". If we stick to your example, you would need to specify in some way that in Python * has higher precedence than +, while in Smalltalk it doesn't. You could write some code in Python or Smalltalk or C or Assembler or even for a Turing machine to do that. Or you can write the grammar in some formal language (e.g. BNF) and let the compiler interpret that (or generate native code from it and run that). The latter is what a "universal compiler" would do and parser generators have been around for a long time (they were nothing new when I studied CS in the 1980's). While there is still progress here (I've come across a few interesting papers over the last year or so), I'd say that part of the problem is solved. The second part is converting a parse tree into code. I am quite sure that it is possible to devise a formal language to specify the semantics of any programming language and then to use this to generate the code. However, I strongly suspect that such
RE: Apriori Algorithm
Speaking for myself, that is a very significant piece of homework to do and unless you do quite a bit and get stuck and ask for help in some aspect, this is not the place to ask if anyone wishes to do all the work. The assignment seems to want you to write your own code to implement the algorithm. So unless the outlines of the algorithm were taught in class or a textbook, you need to find the algorithm by searching the internet and you may find places with more or less of a good description: https://en.wikipedia.org/wiki/Apriori_algorithm Once you find one you need to choose a language you are able to program in from the list of choices and see what libraries of allowed components are likely to be helpful so you do not have to start from nothing but the basics. If you decide on JAVA or C++, this is not the forum to ask. As I noted earlier, your data will need to be read in by any program and made into whatever format your algorithm uses. What this is will be up to you but note you have a collection of collections where you are told of two very specific datasets that must be handled by the algorithm. So your algorithm might be asked to just handle interest, or perhaps also character strings and arbitrary items. You need to know what is expected so you design your code to handle it. Given the nature of the data you show, it should not use anything that holds a fixed number of items as the number of items per line varies and presumably it should handle any number of such lines. It looks like for a basic version they suggest a particular parameter be included and they want an extended version that does more and another that does even more. Looks like a serious amount of work and way too detailed to expect anything but this kind of answer. Many people get paid hundreds of dollars per hour to do things like this and if you learn how, you could be one. Presumably your class has taught you most of what is needed to understand the assignment and how to find out more. I, like most others here, have not been in the class or had any reason to think about this problem before. Any voluntary role here is generally to help with questions about fairly specific python code as compared to big projects. Good luck! -Original Message- From: Python-list On Behalf Of sarang shah Sent: Sunday, March 7, 2021 5:23 PM To: python-list@python.org Subject: Re: Apriori Algorithm On Sunday, March 7, 2021 at 11:55:21 AM UTC-6, Avi Gross wrote: > I apologize for my earlier snide remark as I was not then aware there > was an algorithm called apriori based on the Latin term and wondered > if someone was pulling someone's leg, in advance. > > Someone has posted a pointer to Python code that is supposed to do > that. If that is suitable, then the serious task is to read what they say and use it. > I note the data you show below is ambiguous and not in the format that > code asks for. > > Are you saying each line I see of what looks like integers is a > grouping you want to contrast to other such lines in the algorithm? I > doubt it can be used as is but needs to become some kind of data > structure such as a list of tuples or whatever the algorithm wants. > -Original Message- > From: Python-list > On Behalf Of dn via Python-list > Sent: Sunday, March 7, 2021 3:09 AM > To: pytho...@python.org > Subject: Re: Apriori Algorithm > > On 07/03/2021 20.56, sarang shah wrote: > > I have this dataset in a text file I need to make an apriori > > algorithm > based on it. Please help. > > > > 25 52 164 240 274 328 368 448 538 561 630 687 730 775 825 834 > > 39 120 124 205 401 581 704 814 825 834 > > 35 249 674 712 733 759 854 950 > > 39 422 449 704 825 857 895 937 954 964 > > 15 229 262 283 294 352 381 708 738 766 853 883 966 978 > > 26 104 143 320 569 620 798 > > 7 185 214 350 529 658 682 782 809 849 883 947 970 979 > > 227 390 > > 71 192 208 272 279 280 300 333 496 529 530 597 618 674 675 720 855 > > 914 932 > > > 183 193 217 256 276 277 374 474 483 496 512 529 626 653 706 878 939 > > 161 175 177 424 490 571 597 623 766 795 853 910 960 > > 125 130 327 698 699 839 > > 392 461 569 801 862 > > 27 78 104 177 733 775 781 845 900 921 938 > > 101 147 229 350 411 461 572 579 657 675 778 803 842 903 > > 71 208 217 266 279 290 458 478 523 614 766 853 888 944 969 > > 43 70 176 204 227 334 369 480 513 703 708 835 874 895 > > 25 52 278 730 > > 151 432 504 830 890 > Great! > For what purpose - is this a 'homework assignment'? > What code do you have so far? > > -- > Regards, > =dn > -- > https://mail.python.org/mailman/listinfo/python-list Yes. Each line is a group set. a.Pleaseimplement analgorithm of your choice (Apriori, or FP-Tree) tofind frequent itemsets. You can importstandardlibraries/modules, but thealgorithm should be written by yourself. Min-support should be a user input parameter. Two sample of transaction datasets, Dataset1and Dataset2, are providedto test if your program works correctly.You can usePyt
RE: .title() - annoying mistake
Speaking for myself, I am beyond tired of this topic, however informative parts have been. I will say it is irrational to try to impose rationally across all possible languages, let alone people like me who often combine 3 or more language in a single sentence when talking to others like myself with a dynamic to nonsensical grammar. Try capitalizing someone's name when they insist on being known by a purely numerical name like ..., or just 7 of 9 or even !@zq. So, yes, a purist might want a truly comprehensive piece of code larger than say all the code in Microsoft WORD, that first figures out what language might be used and uses locale and guesswork to even try to narrow down that the words are meant to be treated as they are in Bavaria and see if the use is informal or say adhering to the rules for submitting in APA format or whatever. We are talking about an extremely hard problem that will still likely make mistakes if I am from Bavaria but submitting a science fiction article mimicking what I imagine might be the way it will be written in 3010. So I suggest we take the source for the title function and rename it something enlightening like changeCaseRandomMethod42dInitialUpperElseLower() so it can be used mysteriously by anyone that wants. Then deprecate the use of title() while keeping that as some form of alias for the fantastic new name. Anyone wishing to have title mean anything else, feel free to redefine it. But that jokingly does not mean there is no room for improvement. As an example, people often would like words like "FBI" to be left alone and not changed to "Fbi" or in some other contexts, not be flagged as a spelling error let alone corrected. I suspect the same or relate functions might accept an optional arguments like maintainAllCAPS=TRUE so FBI is left alone, albeit LOTUS123 might become Lotus123 or not. I have seen setups where a programmer makes every imaginable function they can think of but at some later point, some profiling of actual usage shows that 80% of them are NEVER used. Often, that is because nobody reads all the documentation to find out if it even exists or there is a simple workaround. If the only thing bothering you is that a small list of words like FBI comes out wrong, it is simple enough to write a function that post-processes the result of title() and changes those words back. If you designed a brand new language core today, you may indeed want to leave title() out of the core but include it as an optional module, perhaps with a more descriptive name. Tell me if the base should have functions called camelCase(), PascalCase(), arrayHungarianCase() or underscore_case() -Original Message- From: Python-list On Behalf Of Christian Gollwitzer Sent: Monday, March 22, 2021 4:21 PM To: python-list@python.org Subject: Re: .title() - annoying mistake Am 22.03.21 um 16:03 schrieb Robert Latest: > Chris Angelico wrote: >> Cool thing is, nobody in Python needs to maintain anything here. > > That's great because I'm actually having trouble with sending log > messages over the socket conection you helped me with, would you mind having a look? You misunderstood the "here". (native German as well, but I think it means "for keeping .title() up to date) I agree with Chris that .title() can be useful for "title-casing" a single character, whatever that means. It should be documented, though, that it is not suitable to title-case a string, not even in English. Christian -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: convert script awk in python
Alberto,
To convert any algorithm to python (or anything else) you have to understand
it. Do you know what AWK is doing? And does the darn thing work already in
awk? Why do you need to convert it? My suspicion is that it has errors and
if so, it is NOT about converting at all.
I will not solve this for you except to outline what it does.
AWK is designed to read in data and organize it into fields based on rules.
It then sees patterns to match against each clump of data (usually a line)
and performs actions.
The beginning of your shown program simply defined helper functions before
the main program begins. You will need to translate them individually. I
would not have written and formatted it the way you show.
function sq(x) {
return x * x;
}
function dist(x1, y1, z1, x2, y2, z2) {
return sqrt(sq(x1 - x2) + sq(y1 - y2) + sq(z1 - z2)); }
function print_distances() {
if (na == 0)
print "No type 8 atoms.";
else {
min = 1000;
for (a = 0; a < na; a++) {
d = dist(x, y, z, pos[a,"x"], pos[a,"y"], pos[a,"z"]);
#printf "%7.5f ", d;
if (d < min) min = d;
}
printf "%6i%7.5f\n", istep, min;
x = y = z = 0;
delete pos;
na = 0;
}
}
OK, so far? You need to make changes in the above as needed or completely
redo the algorithms in python but first you must recognize what the above
are.
Now the main AWK call is hidden and looks like:
awk 'PROGRAM' $1.lammpstrj > $1_mindist.txt
The above says that AWK is called with a filename the shell script makes
using an argument to the script. The shell also sends the standard output
from awk to a second file with a name based on the same argument. So your
python program needs to open the same file but write to standard output OR
you can rewrite it any way you please to get the same result.
The rest of the program is patterns and actions. But AWK is doing all kinds
of things for you invisibly that you now need to do explicitly in python.
You need to open a file and read a line at a time in a loop. The line must
be parsed into fields the same way AWK would have done. There probably is a
simple package that can do that or you do it by hand.
Now the rest of the program is patterns like:
$1 == 113
$8 == 10
And a more complex one I defer. The above has an action it wants to do for
any line that has a first part exactly equal to 113. The second wants an 8th
part equal to 10. In python, once you have figured out what those parts are
(as well as the other parts) you need to test using something like an "if"
statement for that condition and do what action follows. Here is the
condition then action for the first clause:
$1 == 113 {
if (x || y || z)
print "More than one type $8 atom.";
else {
x = $2; y = $3; z = $4;
istep++;
}
}
I am a tod concerned as to where any of the variables x, y or z have been
defined at this point. I have not seen a BEGIN {...} pattern/action or
anywhere these have been initialized but they are set in a function that as
far as I know has not been called. Weird. Maybe awk is allowing an
uninitialized variable to be tested for in your code but if so, you need to
be cautious how you do this in python.
Be that as it may, I am explaining what I think I see. PATTERN ACTION.
So the next pattern/action is:
$8 == 10 {
pos[na,"x"] = $2; pos[na,"y"] = $3; pos[na,"z"] = $4;
na += 1;
}
And the next one is a bit more complex:
/^ITEM: ATOMS/ && na != 0 { print_distances(); }
It does pattern matching on the original line and asks for the text being
looked for to be at the beginning of the line. So you need to learn how to
ask python to match that pattern. And note the && joins two Boolean parts.
All the above patterns are checked for on every line and you need to know
what AWK does once it matches. Does the code above ask you to move to the
next line when it matches or do every applicable action?
So you have been in a loop and when you reach the end of the file, you need
to get out of the loop. Only then does the final pattern match:
END { print_distances(); }
I hope some of that was helpful. To do the full job you need to do way more
than translate how AWK does something like use curly braces for grouping
while python uses indentations. AWK is designed to be a filter in the ways
described above and does many helpful things behind the scenes. You need to
do all that work for yourself in python.
Of course, having said all that, I know this is a common enough problem that
people have solved by making modules that do awk-like activities in python
but I have no experience there. Had you done a search for something like
this, an answer might have presented itself that involves less work for you,
unless this is homework that needs to be done by you:
https://www.google.com/search?q=python+awk+module&sxsrf=ALeKk03gD2jZYJkZ0cGv
zbKlErWzQJ5Spw%3A1616510303610&source=hp&ei=X_1ZYJPDIob
RE: convert script awk in python
Cameron,
I agree with you. I first encountered AWK in 1982 when I went to work for
Bell Labs.
I have not had any reason to use AWK since before the year 2000 so I was not
sure that unused variables were initialized to zero. The code seemed to
assume that. I have learned quite a few languages since and after a while,
they tend to blend into each other.
I think it would indeed have been more AWKthonic (or should that be called
AWKward?) to have a BEGIN section in which functions were declared and
variables clearly initialized but the language does allow some quick and
dirty ways to do things and clearly the original programmer used some.
Which brings us back to languages like python. When I started using AWK and
a slew of other UNIX programs years ago, what I found interesting is how
much AWK was patterned a bit on the C language, not a surprise as the K in
AWK is Brian Kernighan who had a hand in C. But unlike C that made me wait
around as it compiled, AWK was a bit more of an interpreted language and I
could write one-liner shell scripts (well, stretched over a few lines if
needed) that did things. True, if you stuck an entire program in a BEGIN
statement and did not actually loop over data, it seems a tad wasteful. But
sometimes it was handy to use it to test out a bit of C code I was writing
without waiting for the whole compile thing. In a sense, it was bit like
using the python REPL and getting raid feedback. Of course, when I was an
early adopter of C++, too many things were not in AWK!
What gets me is the original question which made it sound a bit like asking
how you would translate some fairly simple program from language A to
language B. For some fairly simple programs, the translation effort could be
minimal. There are often trivial mappings between similar constructs. Quite
a bit of python simply takes a block of code in another language that is
between curly braces, and lines it up indented below whatever it modifies
and after a colon. The reverse may be similarly trivial. There are of course
many such changes needed for some languages but when some novel twist is
used that the language does not directly support, you may need to innovate
or do a rewrite that avoids it. But still, except in complicated
expressions, you can rewrite x++ to "x += 1" if that is available or "x = x
+ 1" or "x -> x + 1" or whatever.
What gets me here is that AWK in his program was being used exactly for
what it was designed. Python is more general-purpose. Had we been asked (not
on this forum) to convert that AWK script to PERL, it would have been much
more straightforward because PERL was also designed to be able to read in
lines and break them into parts and act on them. It has constructs like the
diamond operator or split that make it easy.
Hence, at the end, I suggested Tomasz may want to do his task not using just
basic python but some module others have already shared that emulates some
of the filter aspects of AWK. That may make it easier to just translate the
bits of code to python while largely leaving the logic in place, depending
on the module.
Just to go way off the rails, was our annoying cross-poster from a while
back also promising to include a language like AWK into their universal
translator by just saving some JSON descriptions?
-Original Message-
From: Python-list On
Behalf Of Cameron Simpson
Sent: Tuesday, March 23, 2021 6:38 PM
To: Tomasz Rola
Cc: Avi Gross via Python-list
Subject: Re: convert script awk in python
On 23Mar2021 16:37, Tomasz Rola wrote:
>On Tue, Mar 23, 2021 at 10:40:01AM -0400, Avi Gross via Python-list wrote:
>[...]
>> I am a tod concerned as to where any of the variables x, y or z have
>> been defined at this point. I have not seen a BEGIN {...}
>> pattern/action or anywhere these have been initialized but they are
>> set in a function that as far as I know has not been called. Weird.
>> Maybe awk is allowing an uninitialized variable to be tested for in
>> your code but if so, you need to be cautious how you do this in python.
>
>As far as I can say, the type of uninitialised variable is groked from
>the first operation on it. I.e., "count += 1" first initializes count
>to 0 and then adds 1.
>
>This might depend on exact awk being used. There were few of them
>during last 30+ years. I just assume it does as I wrote above.
I'm pretty sure this behaviour's been there since very early times. I think
it was there when I learnt awk, decades ago.
>Using BEGIN would be in better style, of course.
Aye. Always good to be up front about initial values.
>There is a very nice book, "The AWK Programming Language" by Aho,
>Kernighan and Weinberger. First printed in 1988, now free and in pdf
>format. Go search.
Yes, a really nice book. [... walks into the other room to get his copy ...]
October 1988.
Wow. There&#x
RE: convert script awk in python
Alan,
Back when various UNIX (later also included in other Operating environments
like Linux and the Mac OS and even Microsoft) utilities came along, the
paradigm was a bit different and some kinds of tasks were seen as being done
with a pipeline of often small and focused utilities. You mentioned SED
which at first seems like a very simple tool but if you look again, it can
replace lots of other tools mostly as you can write one-liners with lots of
power. AWK, in some sense, was even more powerful and can emulate so many
others.
But it came with a cost compared to some modern languages where by attaching
a few modules, you can do much of the same in fewer passes over the data.
I am not sure if I mentioned it here, but I was once on a project that
stored all kinds of billing information in primitive text files using a
vertical bar as record separator. My boss, who was not really a programmer,
started looking at analyzing the data fairly primitively ended up writing
huge shell scripts (ksh, I think) that remotely went to our computers around
the world and gathered the files and processed them through pipelines that
often were 10 or more parts as he selectively broke each line into parts,
removed some and so on. He would use /bin/echo, cut, grep, sed, and so on.
The darn thing ran for hours which was fine when it was running at midnight
in Missouri, but not so much when it ran the same time in countries like
Japan and Israel where the users were awake. I got lots of complaints and
showed him how his entire mess could be replaced mostly by a single AWK
script and complete in minutes.
Of course, now, with a fast internet and modern languages that can run
threads in parallel, it probably would complete in seconds. Maybe I would
have translated that AWK to python after all, but these days I am studying
Kotlin so maybe ...
As I see it, many languages have a trade-off. The fact that AWK decided to
allow a variable to be used without any other form of declaration, was a
feature. It could easily lead to errors if you spelled something wrong. But
look at Python. You can use a variable to hold anything just by using it. If
you spell it wrong later when putting something else in it, no problem. You
now have two variables. If you try to access the value of a non-initialized
variable, you get an error. But many more strongly-typed languages would
catch more potential errors. If you store an int in a variable and later
mistakenly put a string in the same variable name, python is happy. And that
can be a GOOD feature for programmers but will not catch some errors.
Initializing variables to 0 really only makes sense for numeric variables.
When a language allows all kinds of "objects" you might need an
object-specific default initialization and for some objects, that makes no
sense. As you note, the POSIX compliant versions of AWK do also initialize,
if needed, to empty strings.
But I wonder how much languages like AWK are still used to make new programs
as compared to a time they were really useful. So many people sort of live
within one application in a GUI rather than work at a textual level in a
shell where many problems can rapidly be done with a few smaller tools,
often in a pipeline.
Avi
-Original Message-
From: Python-list On
Behalf Of Alan Gauld via Python-list
Sent: Wednesday, March 24, 2021 5:28 AM
To: python-list@python.org
Subject: Re: convert script awk in python
On 23/03/2021 14:40, Avi Gross via Python-list wrote:
> $1 == 113 {
> if (x || y || z)
> print "More than one type $8 atom.";
> else {
> x = $2; y = $3; z = $4;
> istep++;
> }
> }
>
> I am a tod concerned as to where any of the variables x, y or z have
> been defined at this point.
They haven't been, they are using awk's auto-initialization feature.
The variables are defined in this bit of code. The first time we see $1 ==
113 we define the variables. On subsequent appearances we print the warning.
> far as I know has not been called. Weird. Maybe awk is allowing an
> uninitialized variable to be tested for in your code but if so, you
> need to be cautious how you do this in python.
It's standard behaviour in any POSIX compliant awk, variables are
initialised to empty strings/arrays or zero as appropriate to first use.
The original AWK book has already been mentioned, which covers nawk.
I'll add the O'Reilly book "sed & awk" which covers the POSIX version and
includes several extensions not covered in the original book. (It also
covers sed but that's irrelevant here)
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: convert script awk in python
Just to be clear, Cameron, I retired very early and thus have had no reason to use AWK in a work situation and for a while was not using UNIX-based machines. I have no doubt I would have continued using WK as one part of my toolkit for years albeit less often as I found other tools better for some situations, let alone the kind I mentioned earlier that are not text-file based such as databases. It is, as noted, a great tool and if you only had one or a few tools like it available, it can easily be bent and twisted to do much of what the others do as it is more programmable than most. But following that line of reasoning, fairly simple python scripts can be written with python -c "..." or by pointing to a script Anyone have a collection of shell scripts that can be used in pipelines where each piece is just a call to python to do something simple? -Original Message- From: Cameron Simpson Sent: Wednesday, March 24, 2021 6:34 PM To: Avi Gross Cc: python-list@python.org Subject: Re: convert script awk in python On 24Mar2021 12:00, Avi Gross wrote: >But I wonder how much languages like AWK are still used to make new >programs as compared to a time they were really useful. You mentioned in an adjacent post that you've not used AWK since 2000. By contrast, I still use it regularly. It's great for proof of concept at the command line or in small scripts, and as the innards of quite useful scripts. I've a trite "colsum" script which does nothing but generate and run a little awk programme to sum a column, and routinely type "blah | colsum 2" or the like to get a tally. I totally agree that once you're processing a lot of data from places or where a shell script is making long pipelines or many command invocations, if that's a performance issue it is time to recode. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
RE: convert script awk in python
Michael,
A generator that opens one file at a time (or STDIN) in a consistent manner,
would be a reasonable thing to have as part of emulating AWK.
As I see it, you may want a bit more that includes having it know how to
parse each line it reads into some version of names that in Python might not
be $1 and $2 types of names but may be an array of strings with the complete
line perhaps being in array[0] and each of the parts.
Clearly you would place whatever equivalent BEGIN statements in your code
above the call to the generator then have something like a for loop
assigning the result of the generator to a variable and your multiple
condition/action parts in the loop. You then have the END outside the loop.
But it is far from as simple as that to emulate what AWK does such as
deciding whether you stop matching patterns once the first match is found
and executed. As I noted, some AWK features do not line up with normal
python such as assuming variables not initialized are zero or "" depending
on context. There may well be scoping issues and other things to consider.
And clearly you need to do things by hand if you want a character string to
be treated as an integer, ...
But all fairly doable, albeit not sure an easy translation between an AWK
script into python is trivial, or even a good idea.
You could do a similar concept with other utilities like sed or grep or
other such filter utilities where the same generator, or a variant, might
automate things. I am pretty sure some module or other has done things like
this.
It is common in a language like PERL to do something like this:
while(<>)
{
# get rid of the pesky newline character
chomp;
# read the fields in the current record into an array
@fields = split(':', $_);
# DO stuff
}
The <> diamond operator is a sort of generator that reads in a line at a
time from as many files as needed and sticks it in $_ by default and then
you throw away the newline and split the line and then do what you wish
after that. No reason python cannot have something similar, maybe more
wordy.
Disclaimer: I am not suggesting people use AWK or PERL or anything else. The
focus is if people come from other programming environments and are looking
at how to do common tasks in python.
-Original Message-
From: Python-list On
Behalf Of Michael Torrie
Sent: Friday, March 26, 2021 8:32 PM
To: python-list@python.org
Subject: Re: convert script awk in python
On 3/25/21 1:14 AM, Loris Bennett wrote:
> Does any one have a better approach?
Not as such. Running a command and parsing its output is a relatively
common task. Years ago I wrote my own simple python wrapper function that
would make it easier to run a program with arguments, and capture its
output. I ended up using that wrapper many times, which saved a lot of
time.
When it comes to converting a bash pipeline process to Python, it's worth
considering that most of pipelines seem to involve parsing using sed or awk
(as yours do), which is way easier to do from python without that kind of
pipelining. However there is a fantastic article I read years ago about how
generators are python's equivalent to a pipe.
Anyone wanting to replace a bash script with python should read this:
https://www.dabeaz.com/generators/Generators.pdf
Also there's an interesting shell scripting language based on Python called
xonsh which makes it much easier to interact with processes like bash does,
but still leveraging Python to process the output.
https://xon.sh/ .
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: convert script awk in python
https://docs.python.org/3/library/fileinput.html Dan, Yes, fileinput sounds like what I described and more. It does indeed seem to emulate the interface in programs like AWK including using "-" as a placeholder for standard input. Now all you need is to have it also do the split! ∀vi ∃. Grθß -Original Message- From: Python-list On Behalf Of 2qdxy4rzwzuui...@potatochowder.com Sent: Friday, March 26, 2021 9:43 PM To: python-list@python.org Subject: Re: convert script awk in python On 2021-03-26 at 21:06:19 -0400, Avi Gross via Python-list wrote: > A generator that opens one file at a time (or STDIN) in a consistent > manner, would be a reasonable thing to have as part of emulating AWK. https://docs.python.org/3/library/fileinput.html -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: python documentation
What are the odds, Chris, that rewriting an existing project written in an older version of a language like python FROM SCRATCH into any other existing language, would be easier than updating it to the same language which made fairly specific changes and has some guidelines how to update? True, if you have programmers already knowing the other language handy, fine. But as I study other languages, I keep finding things they often do invisibly in the compiler or interpreter that make me wonder why anyone thinks they can write one program that reads them all as if with a magic ring. Some features make translations far from straightforward, not that they cannot be done, but some thought is needed and maybe a change is aspects of how the darn thing is built. What you are expressing is the fact that the longer we encourage people to keep using the old, the more painful it is to move forward with the new. At some point, so many changes may accumulate, that catching up may not be worth doing. Any nontrivial program that uses many packages and modules will not find identical things in a new target language, for example. Some nice concise ways some things are done may work differently elsewhere and need to be redesigned completely or lead to lots of errors. Now if the case was being made to switch to a more recent advanced language, maybe. But the languages he suggested strike me as fairly ancient, even if they too have been evolving. As you note, he is free to do what he wishes but not free to force others to help him when it is not in their interest. -Original Message- From: Python-list On Behalf Of Chris Angelico Sent: Saturday, March 27, 2021 1:37 AM To: Python Subject: Re: python documentation On Sat, Mar 27, 2021 at 4:20 PM wrote: > > Chris, > > you seem to imply, that I have compiled said versions without reason > and that the same would be possible on basis of Python 3 - which is > simply not true. Maybe you are not enough acquainted with Qt and > belonging libraries alike PyQtGraph. Maybe you are just not willing to > see / accept these arguments. > > By the way, some months ago I started trying to migrate to Python 3 > and gave up in favor of creating said compilation. Compatibility of > Python and its Packages decreased with V3 significantly. A whole lot > of minor and major incompatibilities between your subversions and > belonging packages. This was one reason, why Java took the route to its own death. FUD. Lots and lots of FUD. More reasons to not promote your distribution. Use it if you will, but it doesn't merit any further visibility. > With a view to the mid and long term future, this discussion even > gives me cause to ponder about whether it doesn't make more sense to > rely more on C# and WinForms for professional projects from now on. I > am fluent in both too and it always makes sense to bet on the right > horse at an early stage. If you prefer, go for it. Everyone claims that it's easier to move to rather than to migrate to Python 3, and I'm calling people's bluffs now. Go ahead and move to another language if it's easier - it's no skin off my nose. Or maybe it isn't easier, and that's just an empty argument. Funny how it keeps coming up. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: New Python curses book
Congratulations, Alan, on the book. I continue to wonder if people will buy the book for the wrong reason or ban it thinking you created an AI snake that creates and spews new CURSES never heard before. The good news is that for those interested you can click on the book image and see the preview of the beginning parts and see it is about programming, hold the curses. -Original Message- From: Python-list On Behalf Of Alan Gauld via Python-list Sent: Tuesday, March 30, 2021 7:12 AM To: python-list@python.org Subject: Ann: New Python curses book I've just published, in Kindle and paperback formats, my book on "Programming curses with Python". https://www.amazon.co.uk/dp/B091B85B77/ (It should be available in most other Amazon stores too) It is a complete rewrite of the Linux Documentation Project's HowTo for the C ncurses library. It has the same structure but all-new text and several new chapters. All specific to the Python curses package. I've kept the prices as low as Amazon permits (and allowing for fluctuations in their costs). As such, I will therefore make a few cents profit per book from sales (full disclosure). However, I felt this was the easiest and most effective way to get it "out there" where it could be found. I hope it's useful to somebody. Special thanks to all who provided me with feedback on its various iterations, especially Alex Kleider and Bob Stepp who stuck with it right to the end. Thanks guys. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Ann: New Python curses book
Bill, I went afterward to the US Amazon Site and the prices are in dollars and ordered the paperback for $5.99, no shipping cost with PRIME but I ordered, paradoxically, an assortment of mice alongside so it would have been free anyway. The Kindle version is $1.49 https://www.amazon.com/Programming-curses-Python-Alan-Gauld-ebook/dp/B091B85B77/ref=sr_1_2?dchild=1&keywords=alan+gauld&qid=1617129871&sr=8-2 The paperback has a different "cover" that says "Welcome to curses!" which sounds ominous. I might have chosen to call it a "Course on curses and cures" ... So depending on what country you are interested in, it may well be there but you need to not use the supplied URL and go there directly. -Original Message- From: Python-list On Behalf Of William Ray Wing via Python-list Sent: Tuesday, March 30, 2021 2:06 PM To: Python Cc: William Ray Wing ; Python Subject: Re: Ann: New Python curses book I’ve ordered the book (physical volume). It will fulfill a need I’ve had for some time. Unfortunately, it is only available in the UK store, so the shipping cost by far outweighs the book’s cost. Hope for other’s sake, it migrates to the other Amazon stores fairly quickly. Thanks, Bill > On Mar 30, 2021, at 7:12 AM, Alan Gauld via Python-list > wrote: > > I've just published, in Kindle and paperback formats, my book on > "Programming curses with Python". > > https://www.amazon.co.uk/dp/B091B85B77/ > > (It should be available in most other Amazon stores too) -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Horrible abuse of __init_subclass__, or elegant hack?
Chris, Now that it is April 2, I have to ask which of the methods for dealing with chocolate is more pythonic and is there a module for that? Next April, can we switch beans and discuss different grades of coffee and which ones are best to shave at home with a potato peeler? I think that would be equally relevant. I will agree that some kinds of pie-thon have chocolate as a middle ingredient but what good is any kind of pie without coffee? Oops, I should have sent this yesterday! Avi -Original Message- From: Python-list On Behalf Of Chris Angelico Sent: Thursday, April 1, 2021 8:00 PM To: Python Subject: Re: Horrible abuse of __init_subclass__, or elegant hack? On Fri, Apr 2, 2021 at 10:43 AM dn via Python-list wrote: > > On 02/04/2021 10.13, Chris Angelico wrote: > > Well, it's a simple matter of chronology. First you have crude oil, > > then time passes, and then you have plastic and residue. It makes > > sense ONLY if you think of it with a specific ordering, which > > implies Python 3.7 or later. > > My anxiety over 'ordering' comes to the fore: if there are multiple > inputs and/or multiple outputs, how can one tell where the former > series ends and the latter begins? > > "Explicit" cf "implicit"? The exact same way. Before time passes, you have all of the inputs; after time passes, you have all of the outputs. (The way the game goes, all inputs are consumed simultaneously and all outputs produced simultaneously, so there's no chronological distinctions between them.) > > Hot chocolate? Ahh, now, that's the other thing I drink a lot of. I > > even have a dedicated mug with a Cadbury logo on it. Sadly, not > > easily purchasable, but this one was a gift from a family member who > > knows full well that I have my priorities straight. > > Cadbury do not make chocolate! > Neither do Hershey, for that matter! > There, now that I've upset most of the English-speaking world... > > PS they use the milk to 'water down' the chocolate! It's much like > buying Easter Eggs: the density of chocolate is much lower for the price. Chocolate comes in tiers. 1: Top tier chocolates - fine chocolates - are made by true artisans, and are generally purchased in smaller quantities due to pricing. (Although I did once buy something like 50kg of Lindor balls. That was a gd day.) 2: Everyday chocolate. Mass-produced, so it's lower quality but far lower in price. Sure, it's not as amazing as eating Guylian or Godiva, but you can eat a block of Cadbury every day and still be within budget. 3: Cooking chocolate. Comes cheaper than typical block chocolate, is usually firmer, and takes more effort to make good use of, but you can take a potato peeler to it and put chocolate shavings on your Pavlova, which doesn't really work as well with a block of Dairy Milk. 4: Cheap chocolate. Easter eggs, bulk stuff, the sorts of things you throw at children to keep them happy. Sometimes has nostalgia value, but it's low-grade stuff. Some of it isn't too bad, but you end up paying MORE for it than you would for tier two chocolate, since it's usually packaged up far too much. Also, some of it is downright disgusting. I won't name names, but one time I had access to a bulk load of white chocolate (and yes, you could say that white chocolate isn't chocolate to start with, but some of it is actually good), and it tasted like milk powder. Why people pay so much for low-grade chocolate wrapped up in fiddly foil covers, I don't know. > View > https://www.whittakers.co.nz/en_NZ/products/72-dark-ghana/block-250g > before questioning my curmugeonly credentials! Oh yes, that's definitely one of the good brands. By "credentials", do you mean that you have some connection with the company? If not, that's fine, but it would certainly be notable if you do! ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Yield after the return in Python function.
Terry: ... '__missing__' is new since I learned Python ...
With so many new dunder variables added, I am wondering when some dunderhead
comes up with:
__mifflin__
The documented use paper is:
https://theoffice.fandom.com/wiki/Dunder_Mifflin_Paper_Company
-Original Message-
From: Python-list On
Behalf Of Terry Reedy
Sent: Monday, April 5, 2021 3:01 PM
To: python-list@python.org
Subject: Re: Yield after the return in Python function.
On 4/5/2021 1:53 PM, Chris Angelico wrote:
> On Tue, Apr 6, 2021 at 3:46 AM Terry Reedy wrote:
>> *While 'a and not a' == False in logic, in Python it might raise
>> NameError. But that would still mean that it is never True, making
>> 'yield 0' still unreachable.
When I wrote that, I knew I might be missing something else.
> And even just the lookup can have side effects, if your code is
> pathologically stupid.
Or pathologically clever.
class Wat(dict):
> ... def __missing__(self, key):
> ... global count
> ... count -= 1
> ... return count
'__missing__' is new since I learned Python. I barely took note of its
addition and have never used it. Thanks for the example of what it can do.
One could also make it randomly return True or False.
count = 2
eval("print(a and not a)", Wat(print=print))
> True
>
> So Python can't afford to treat this as dead code.
This gets to the point that logic and math are usually atemporal or at least
static (as in a frozen snapshot), while computing is dynamic. In algebra,
the canon is that all instances of a variable are replaced by the same
value.
Python *could* do the same for expresssions: load 'a' (in this case) once
into a register or stack slot and use that value consistently throughout the
expression. Replacing the eval with the following exec has the same effect.
exec("tem=a; print(tem and not tem)", Wat(print=print)) # print False
In this example, one could disable the binding with __setitem__ (resulting
in printing 0), but python code cannot disable internal register or stack
assignments.
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: translating external files type thing
https://translate.google.com/?sl=is&tl=en&op=translate Or, you could do it the hard way. Kidding aside, there may be a python module you can hand a file name or contents to and have it do much of the job using some API that may tap into the above Google resource. Dan specifically suggested importing googletrans ... -Original Message- From: Python-list On Behalf Of Quentin Bock Sent: Tuesday, April 13, 2021 3:16 PM To: python-list@python.org Subject: translating external files type thing okay, so I'm making a translating program, and I have files set to be translated (they're in Icelandic) and I want all of them to be read in English but I'm not sure how to open the files and translate them at the same time. I'm also not even sure how to translate an external file to English from another language. I can't show code for this because I don't know if it's possible to do what I'm asking. Hopefully, this question is understandable Thanks -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Current thinking on required options
Sidestepping the wording of "options" is the very real fact that providing
names for even required parts can be helpful in many cases.
There re programs that may not require anything on the command line to be
done but many need something to provide some flexibility.
So, I tend to agree that in many cases you need an explicit or implicit
argument for a program to do something useful. If I have a simple program
that looks up a phone number when given a name, it would require a name
UNLESS the default behavior is to show ALL entries or repeat whatever I
asked for last time by keeping track or provide the number I ask for most or
pick a random one or return silently having done nothing ...
But I want to point out something OBVIOUS. The requested program in my view
HAS NO required arguments! All are in a sense optional but at the same time
are all mandatory in some circumstances.
It can be called two equally valid ways:
grocli [-h]
OR
grocli -o {check,add,delete} -u USERS [USERS ...]] -g GROUP
When called to ask for help, none of the other arguments are required or are
ignored or worse. When called with the second properly formed set of
arguments, in any order, I assume any "-h" is either ignored or an error.
So I would possibly have TWO usage statements and in both cases, NO optional
arguments! Either you ask for help or you provide everything else.
Clearly your actual code can be designed many ways including allowing all
combinations and throwing in help when asked for in addition to doing what
is requested or allowing multiple -u arguments instead of multiple arguments
following a single -u and so forth. Heck, it can perhaps accept random
additional arguments and pass them along to another command it uses
internally without question in a "..." situation.
So a syntax for defining a program as documentation like the above may need
an OR approach or be even more complex when say two or more arguments can be
used but only ONE is allowed and then it may be mandatory. Picture a -m to
suggest units are metric versus ...
And this "-h" notation is very common in programs and can cause the
description of how a program should be used more complex than it needs to be
if you insist on just one line showing how to use it rather than giving
several valid usages.
-Original Message-
From: Python-list On
Behalf Of Dan Stromberg
Sent: Monday, April 19, 2021 12:04 PM
To: Loris Bennett
Cc: Python List
Subject: Re: Current thinking on required options
On Mon, Apr 19, 2021 at 2:55 AM Loris Bennett
wrote:
> However, the options -o, -u, and -g are required, not optional.
>
> The documentation
>
> https://docs.python.org/3/library/argparse.html#required
>
> advises against required options and here
>
>
> https://stackoverflow.com/questions/24180527/argparse-required-argumen
> ts-listed-under-optional-arguments
>
> a way of adding a section 'required arguments' to the usage is
> described.
>
Of _course_ some options need to be required.
I can't imagine what the author of that page was thinking.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: do ya still use python?
Yes, Python is a moving target, as are quite a few such things these days. The changes when release 3 came along mean that what you find by a search may not apply to your situation. And as new features get added, some advice might shift. The presence of so many add-on modules also means that the simplest way to do something is not to do it but use code that has already been done and hopefully has been used enough so many bugs have been ironed out. Modules and packages of various sorts also can change and I often see a neat new feature that is later deprecated as it is replaced by a neater or more abstract version or ... So it is sometimes worthwhile to put humans back into the process after searching for something like how to read in data from some file format. I once wanted to be able to read in data from EXCEL sheets in another language and spent many hours trying various packages a search suggested but oddly, one after another did not work for ME. One, for example, required a JAVA installation different than what I had. Some packages were no longer available or maintained. Some did not easily do what I wanted. Some worked with older versions of the EXCEL file formats. I eventually found one I liked. But a human with experience might have steered me to something up-front that was being used NOW by people with minimal problems and that was compatible. There is of course much to be said for asking people to show they did SOME work before bothering others. I always do that if I can guess at what keywords might help zoom in on useful info. But as the internet continues to grow, there are too many things found that are not helpful especially when some search words have alternate meanings as in other human languages. But perhaps the purpose of some groups varies. If you want a discussion of the best way to do something and what the tradeoffs might be or whether it may be better to use other tools, or which way to add a new feature to your language and so on, you want different kinds of people involved depending on the topic. Alan runs a tutorial group of sorts intended to help people who are often quite new to python. I hung out there a while and realized my personal interests tended to be in other directions than helping people do things in simple basic ways. I was way beyond that and interested in what is more elegant or efficient or uses some novel feature or even how to use modules already designed for it. But for students in a class wanting a little hint for their homework, this is often the wrong approach. Maybe after they have mastered some basics, they might benefit from looking deeper. But at their stage, searching the internet for answers may not work well as they may not even know how to ask the right way or be turned off by some of what they read that assumes they already know much more. So there really is room for many forums and methods and ideally people should try to focus on the ones that work better for what they are doing. -Original Message- From: Python-list On Behalf Of o1bigtenor Sent: Wednesday, April 21, 2021 6:07 AM To: Terry Reedy Cc: Python Subject: Re: do ya still use python? On Tue, Apr 20, 2021 at 6:26 PM Terry Reedy wrote: > > On 4/20/2021 4:32 AM, Alan Gauld via Python-list wrote: > > > We see the same trend on the tutor list, traffic has dropped off by > > a factor of 3-5 times what it was at its peak. And the questions are > > changing too, fewer basic things about loops and writing functions, > > more about specific library modules and such. > > I suspect that at least some such questions have good answers on > StackOverflow that questioners could profitably read first. > Respectfully - - - - I would disagree. I am finding when I'm looking for answers that the generalist sites most often cough up responses - - - - - yes there are responses - - - but those responses are for software of at best 5 to 6 years ago and all too often its for software of 15 + years ago. Most often those 'answers' just aren't applicable anymore. Not saying that there never are answers but I've gotten to including a 'date' in my searching and then there are a not less links proffered by the search engine! HTH -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Proposal: Disconnect comp.lang.python from python-list
Chris, Given some notice, what stops anyone from joining the mailing list before there is a divorce between the forums? Everybody has trivial access to an email account these days and many mailers allow incoming messages that fit a pattern to be placed in some names folder to be read as a group if one wishes. The difference would include less spoofing but also the ability to remove and that are causing annoyance here. Avi -Original Message- From: Python-list On Behalf Of Chris Angelico Sent: Wednesday, May 5, 2021 8:37 PM To: Python Subject: Re: Proposal: Disconnect comp.lang.python from python-list On Thu, May 6, 2021 at 10:32 AM Paul Bryan wrote: > > Given the ease of spoofing sender addresses, and its propensity for > use in anonymous spamming and trolling (thanks python-list-owner for > staying on top of that!), I propose to disconnect comp.lang.python > from the python-list mailing list. Both would then operate independently. > As someone who exclusively follows the mailing list, I selfishly want to support this, as it would notably improve the signal-to-noise ratio. But I'm aware that there are a number of good people who currently use the newsgroup, and those people would then be abandoned in the cess-pool. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: neoPython : Fastest Python Implementation: Coming Soon
Chris, I got the fastest python yesterday as it was so fast that it beat the one coming tomorrow. The trick is adding back the legs that evolution allowed to lapse. Without bated breath, Regardless, Avi -Original Message- From: Python-list On Behalf Of Chris Angelico Sent: Wednesday, May 5, 2021 12:02 PM To: Python Subject: Re: neoPython : Fastest Python Implementation: Coming Soon On Thu, May 6, 2021 at 2:01 AM Mr Flibble wrote: > > neoPython : Fastest Python Implementation: Coming Soon > > Message ends. > > /Flibble > My breath: not being held. Message ends. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Programmed energy use
Benjamin, The topic is being changed by me because I get amused by statistics. We can all agree that any form of energy use uses energy, duh! But that does not make it good or bad in isolation. If I had a job that had me wake up every day in the dark, drive for an hour to an airport, hop across the country (perhaps in a private plane) and drive a rental from there to work for another hour, find a few hours to do some work, then repeat and head home, that would indeed be wasteful as it happened several hundred times a year. If I opted to use a PC with ZOOM and a few other tools instead, including using resources in the cloud and especially programs running as PYTHON, would it be a net good thing for the environment? The reality is the success of Python as an environment that is used in many servers, often alongside others, simply means it is used MORE. If no other language was ever used, it would get all the blame. If it was rarely used, it might hardly matter if it was efficient or not. As it happens, I doubt it is anywhere as inefficient as some suggest. Some of the other alternatives arguably are much worse. Think how fast programs run that are written as humungous shell scripts that start a new full-process for each minor task, for example. One potential set of solutions is fairly clear. Some programs, in ANY language, perform differently depending on which algorithms are used and many other such factors. So sure, a substantial number of programs of any type that are run frequently probably could be evaluated to make them more efficient. A dumb example is languages that allow you to keep adding entries to something like a vector or list or data.frame and do so by making a new data structure each time of one length longer and COPYING all the items over and then allowing the previous one to be garbage collected. Boy can that become expensive for longer input or calculations. But if you KNOW how many items you may need, you can reserve enough space in advance and fill it as you go along, or other such variants. In my reality, many languages are now hybrids with some parts implemented using libraries already written carefully for efficiency and often in languages like C or C++ or FORTRAN or perhaps even lower-level languages that are machine-specific. Proper use of these routines gives a nice boost. And, sure, if you run some programs that never use many language features on a server, I suspect you could make a customized version of something like Python built that does not incorporate much of what will never be used so the core loaded is smaller and perhaps in many ways faster. I know that has been done for example for some smaller processors. Having said all that, if someone really made a truly compliant improvement on python that ran everything and was compatible with future additions, great. Would it be public domain and open-code? Would it be handed over to the groups currently responsible for python so parts or the whole could be used if people were interested? Would it be maintained? Free? ... My impression from the past is that what is being offered will NOT be a python interpreter at all but rather a file or files that define the language and an engine that takes programs in any language that has been defined that way and combining the two to compile some kind of executable. Some of that has sort of been done, albeit just for one language at a time. Adding features to the language though, or even fixing any errors, might become a bit weirder. But none of this really has to do with the environment really. The goal for many people is to make our computing CHEAPER and more reliable and so on. Obviously faster processors or peripherals may deliver more bang for the buck and reducing the heat generated or energy used (somewhat related) are other such facets. But if we improved our computers (including software like python) and simply found lots more uses for them so we kept adding more farms of servers such as to mine new bitcoin or whatever, we would not have progress. Heck, we can cut back on wasting energy by not allowing children to play video games or look at social media for more than an hour a day and limit people from getting their email more than once a day like in the old days! For adults, we could remove most pornography which apparently is one of the biggest uses for the internet. Does anyone have even a guess on what globally is contributing most to such energy usage even within the computer realm? I mean can we blame Apple and Microsoft and Google and such for their operating systems which run all the time and often consume all available CPU when nothing else is happening and it just keeps looking for something to do? How much time is spent by SQL queries whether called from python or any other languages, or google searches that may trigger a parallel search in thousands or more places then telling us the first 10, plus s
RE: Bloody rubbish
Actually, Joe, putting in any serious program using toggle switches without anything like a BACKSPACE was very hard as I often had to abort and start again. Doing it twice the same way, Argh Luckily, I only had to do it a few times to learn just like I had to write assembler programs or feed in programs from paper tape or from punch cards. Most of us have moved on stage by stage and now tools like Python or libraries and modules often at higher levels are more the norm. Can you imagine taking any modern program in digital form as zeroes and ones and entering it by hand? Some are huge and especially if anything like shared libraries also has to be keyed in. But reminiscing is getting away from the point of expressing our sarcasm about one of the people that makes us want to segregate this forum as one way to not get into silly discussions like this! -Original Message- From: Python-list On Behalf Of Joe Pfeiffer Sent: Thursday, May 6, 2021 3:03 PM To: python-list@python.org Subject: Re: Bloody rubbish Skip Montanaro writes: >> >> Machine language is so much simpler, and you can code with just a hexpad. >> > > Pshaa... All you need are front panel switches. ;-) (Yes, I had a > professor who required is to 'key' in our programs on the front panel, > of a rack mounted PDP-11 as I recall. Needless to say, we didn't use > an assembler either. We just wrote raw opcodes and their arguments on > paper. This was in the late 70s.) That's right about whn I had to do that for one assignment (on a Nova). Hand-assembling, toggling in, and debugging a program on the front panel was a valuable learning exercise. Doing it a second time wouldn't have been helpful... One nice thing was the computer had core memory, and the students made an agreement as to who got which part. You could work for a while, shut the machine down, come back the next day, power it up, and your program would still be there. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: learning python ...
I have studied many programming languages and am amused when people attack
python as if every other language is somehow more optimal.
Cameron and others have provided examples but look at positives AND negatives.
Yes code like: num = int(num)
does look a tad off as it reuses the same name for something that is actually
in many ways different. You could easily use a new name. But then you would
have TWO variables in your name space and the old one would not be garbage
collected unless you explicitly removed it. If you follow this path, would you
suggest not writing: X = X + 1 either?
It is actually a fairly common practice in many languages to have code like
this:
Var = read in something from a file and make some structure like a data.frame
Var = remove some columns from the above thing pointed to by Var
Var = make some new calculated columns ditto
Var = remove some rows ...
Var = set some kind of grouping on the above or sort it and so on.
As you go along you may keep transforming but have no need for earlier results,
just the new and improved one. So why keep changing names? Some languages
support not having any names for intermediate results by using nested function
calls or pipelines.
The reality is that most languages have a series of what I loosely would call
environments or name tables and as they run, new ones regularly get created and
removed and even the order of them may change. The algorithm for searching for
a name varies and can be in some sense linear or more complex. When looking in
context for a function name, you may follow a different trail or the same one
as for a variable holding a string and in some implementations the same name
shadows and in others does not. Wait till you try to figure out the diamond
pattern of inheritance when you create classes that depend on multiple other
classes ad nauseum and you try to call a method and it tries to find the one
you wanted by searching backwards in an interesting way. Many other languages
decided to just not allow multiple inheritance!
How can you write a recursive function without this kind of variable shadowing?
Each invocation of a function places the internal namespace in front of the
parent so the meaning of a variable name used within is always backed by all
the iterations before it. But with some kinds of closures, a function may be
gone and yet variables it had within it persists. Lots of odd examples more
make this a complex set of CHOICES.
So what if you suggest we allow re-use of names but WARN you. OK, Python is a
largely interpreted language. Normally if you ask to use a variable called X,
it starts a search in the nearest place then the next one and so on till it
finds it or fails. In many programs, variables are fairly local and found
easily. But if you want, you can search dozens or hundreds of places and find
each and every use of X visible at THIS moment and tell the user there are 82
places it can be seen and here they are. Now what? The first 50 places may be
in other instances of the recursive function and you have already been warned
this way 49 times and soon will be warned a few more as it continues to recurse
as it travels down a tree or graph structure quite legitimately. Some of the
other places X may be in use are in some package in a part you are not even
using indirectly or in middle of a for-loop as a token variable and so on. I
suspect that 99.99% of the time re-using a name has no negative consequence.
Making someone keep choosing names like X12346 because there is somewhere an
X12345 seems a tad silly. But why go to all that expense at run-time versus in
some lint program?
I recently studied a language called GO that goes to some length to protect
the user from themselves and often at great expense to getting programming
done. It won't compile a program if it declares a variable and does not use it.
Fair enough but I often want to write a sort of shell of a program and check as
I go to make sure it works before adding more. If I will need 5 variables, I
might declare them up-front then work on interactions and calculations that
only include some of them and later I plan on connecting the rest. Nope, it
won't let me unless I use it as in a print statement or comment it out or just
remove it. Ah, but then it is not happy I did not initialize it so I set it to
zero or something. Later, when I finally use it as intended, I need to remove
the junk.
Some of this annoyance is so common that they had to come up with a way to shut
it up. Consider this line:
Result, err = functionA(args)
It returns two arguments. Sometimes it seems silly to worry about the error but
you MUST write a line of code like:
If (err != nil) {
Do_something
}
Other times you just care if it worked and don't want the result. So placing an
_ for one or the other is a signal to shut the darn compiler up that you are
DELIBERATELY ignoring the return value you did not w
