Python 3.9 pidfd_open

[Weatherby,Gerard]

python introduced os.pidfd_open(), which works as documented.

My development environment, PyCharm, complains about it being undefined.

Should it be in https://raw.githubusercontent.com/python/cpython/3.9/Lib/os.py ?

--
Gerard Weatherby | Application Architect
NMRbox | NAN | Department of Molecular Biology and Biophysics | UConn Health
263 Farmington Avenue, Farmington, CT 06030-6406
uchc.edu
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Object in List : how?

[Weatherby,Gerard]

“Private” properties are more simply / commonly designated by sticking an _ in 
front of the name.

class Node:

def __init__(self,a)
self._a = a

I recommend you read https://docs.python.org/3/tutorial/classes.html.


That’s not to say properties don’t have their uses, but making things “private” 
isn’t a major one.

—
Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular 
Biology and Biophysics
 UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
On Jul 25, 2022, 10:36 AM -0400, Khairil Sitanggang , wrote:
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Thank you everyone. The specific requirements for that class:

*(1)* Provide the values of the "input" (via constructors).
*I think everyone agrees with the way it is implemented in the
example. *

*(2)* Provide other products such as *b* that can be of any type (array,
object, etc.). It is like an "output" if you will.
*I think everyone suggests that it should be designed such that people
should not be able (inadvertently) to change the value from outside the
class. I agree, as a matter of fact this is my intent as well.*

*(3)* About the function calc(), my intent is NOT to expose it outside the
class: it is "private".
*And I see the way to do that from your suggestions. *

Based on all your valuable suggestions, I should be able to accomplish the
3 goals.

By the way, I am writing this code for finite element analysis (FEA):
number crunching. Even though the final goal is to get the correct results,
I still want to write it following the correct python "grammar" and style.

Best regards,
-Irfan

On Mon, Jul 25, 2022 at 3:54 AM Peter Otten <__pete...@web.de> wrote:

On 25/07/2022 02:47, Khairil Sitanggang wrote:
Regarding your comment : "
*However, usually object creation and initialization iscombined by
allowing
arguments to the initializer:*" , so which one of the two classes Node1,
Node2 below is more common in practice? Option 2, I guess.
Thanks,


# option 1:
class Node1:
def __init__(self, a):
self.a = a
self.b = self.calculation()

def calculation(self):
r = self.a + 10
return r

# option 2:
class Node2:
def __init__(self, a, b):
self.a = a
self.b = b

self.b = self.calculation()

def calculation(self):
r = self.a + 10
return r

nd1 = Node1(10)
nd2 = Node2(10, 0) # 0 is dummy, will be overwritten by the call to
calculation()

An attribute that can be calculated from other attributes should never
be modified through other means. Some day you may want b to become
something else, write, for example,

node = Node2(10, "twenty")

and because by then you have forgotten about the calculation() call end
up with a buggy script. But manually invoking the calculation() method
is also bug prone. You have to remember to repeat it every time you
change a:

node = Node1(10)
assert node.b == 20 # OK

node.a = 20
assert node.b == 30 # fails, a and b are out of sync.

The solution Python has to offer is called "property". Properties in
their simplest form are calculated read-only attributes, i. e. when you
write

print(node.b)

under the hood node.a + 10 is calculated. Here's how to change Node1 to
turn b into such a property:

class Node3a:
def __init__(self, a):
self.a = a
def calculation(self):
return self.a + 10
b = property(calculation)

node = Node3a(42)
print(node.b) # 52

node.a = 1
print(node.b) # 11

Often you are not interested in exposing both the calculation() method
and the property. For cases when you only want to access the property
Python provides a way to define the property with a "decorator":

class Node3b:
def __init__(self, a):
self.a = a
@property
def b(self):
return self.a + 10

When you compare the two classes you can see that I

(1) renamed calculation() to b() and

(2) replaced

def b(self): ...
b = property(b)

with

@property
def b(self): ...

thus avoiding the repetitons of the name.

Are there any disadvantages to properties?

What I presented as an advantage, that the value of the attribute is
recalculated every time the attribute is accessed, may sometimes become
a disadvantage, e. g. when it takes a very long time to calculate. In
most cases that should not be a problem, though.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gibw_ZRRGmldOXyvOxYalA9ApszdEvnBdHnTgNVTbgvDAks80irsggsw45ZkniBHx0YRieVHhKAyn1gR$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: random.SystemRandom().randint() inefficient

[Weatherby,Gerard]

Absolutely. The task (“generate a random number”) is ill-defined.

Want a fast random number? Use 552015933 (I just retrieved it from random.org).

Want a true random number, use random.org API. (https://api.random.org/pricing).

Something in between, follow approaches Stefan suggests.



—
Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular 
Biology and Biophysics
 UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
On Jul 26, 2022, 11:45 AM -0400, Stefan Ram , wrote:
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Cecil Westerhof  writes:
I need to get a random integer.

There are all kinds of trade-offs. When you need entropy,
the function sometimes has to collect data from hardware,
which takes some time. Pseudo-random integers sometimes can
be calculated faster. As a compromise, you can get some
entropy and use it to initialize ("seed") a pseudo-random
number generator and then get some pseudo-random numbers
until you seed it again. A linear-congruential generator
should be fast enough, but in some cases might not be random
enough.


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kjJ7U6kkSW8uOTWJCnHeXn0Acczd619asmIrVMA_NOqvP_JMqqQ1Rpp61sptQucbOuEFGb6ot499_FqaRajiS9s$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PEP about recommended project folder layout

[Weatherby,Gerard]

I’m not aware of any standard convention for laying out packages.

PEP 8 (https://peps.python.org/pep-0008/) specifies conventions for how to 
write Python, so a standard layout PEP would not be inconsistent.

—
Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular 
Biology and Biophysics
 UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
On Jul 30, 2022, 4:37 PM -0400, Barry , wrote:
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 30 Jul 2022, at 13:52, c.bu...@posteo.jp wrote:

Isn't there a PEP?

PEP are for improving python. They are not for telling people how to use python.
I would be surprised to fine a PEP that addressed this.

Barry


On 2022-07-26 07:14 c.bu...@posteo.jp wrote:
Hello,

I am not sure if I looked into the correct sources. I was looking in
"PEP 609 – Python Packaging Authority (PyPA) Governance" [1] and the
"PyPA specifications" [2].

My question in short: Is there an official document (e.g. a PEP)
about a recommended layout for project folders.

Looking into the wild and past there are a lot of variations of such
layouts. I am far away from being a pro but depending on experience
in my own projects and what I have learned from others (e.g. in
blog-posts/tutorials) I recommend to have the "test" folder and the
package folder side by side on the same level in the project folder
(the root).

my_project
|- tests
| └ test_*.py
|- my_package
| └ __init__.py
└-- README.md

I sometimes add to it the so called "src"-Layout where the package
folder is one level deeper in an extra "src" folder.

my_project
|- tests
| └ test_*.py
|- src
| └- my_package
| └ __init__.py
└-- README.md

I don't want to discuss the pros and cons of all variations. What I
need is an official document I can use in discussions with other
maintainers. If there is a PEP/document against my current
recommendation I am also fine with this. ;)

Kind
Christian

[1] -- 

[2] -- 


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ipqSQS5_ZFxO37h5h5XALf2DxE3eV-x6qNSrVnKJU3cLZIkLfKhuTzSxi1fb-BvYNVPXeT-nYFX8ogse6_J2Hg$

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ipqSQS5_ZFxO37h5h5XALf2DxE3eV-x6qNSrVnKJU3cLZIkLfKhuTzSxi1fb-BvYNVPXeT-nYFX8ogse6_J2Hg$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dictionary order?

[Weatherby,Gerard]

I don’t see what is surprising. The interface for a dictionary never specified 
the ordering of the keys, so I would not be surprised to see it vary based on 
release, platform, values of keys inserted, number of items in the dictionary, 
etc.





—
Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular 
Biology and Biophysics
 UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
On Aug 1, 2022, 7:48 PM -0400, Dan Stromberg , wrote:
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On Mon, Aug 1, 2022 at 4:42 PM Dan Stromberg  wrote:


Yes, but I'm pretty sure that's been true for a LONG time. The hashes
for small integers have been themselves for as long as I can remember.
But the behaviour of the dictionary, when fed such keys, is what's
changed.

I'm not disputing either of those facts. I'm pointing out that the
apparently arbitrary order of a mapping's keys becomes obvious when you
look at the hashes of those keys.


It looks like the relationship no longer holds at around keys =
list(range(250, 260))

But i == hash(i) holds for the first million values at least.


I could've been more clear. int dict keys stop being stored-in-order at
near 256.

But i == hash(i) holds for the first million values, and probably more.

This suggests to me that there's something more than i == hash(i) going on
inside dict's - but it doesn't much matter what it is for my purposes.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!j3p_Aq5MoGqDk5XMsKb4SKs3U1nfuMOx0wVkSa_hbURJ22w6lP8NrCOc_PYAfELYOdVlC9x6JzLfIMIw5sLe$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Which linux distro is more conducive for learning the Python programming language?

[Weatherby,Gerard]

Just be aware https://docs.python.org/3/ defaults to the latest Python version 
(3.10). When looking up a module, it’s best to explicitly set the documentation 
to the version you are using. It won’t matter the vast majority of the time but 
I have been burned by trying to use a function or parameter that was introduced 
with a later version.

—
Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular 
Biology and Biophysics
 UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
On Aug 4, 2022, 5:00 AM -0400, Turritopsis Dohrnii Teo En Ming 
, wrote:
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On Thu, 4 Aug 2022 at 16:50, dn  wrote:

PS most of us will qualify for RedHat's Developer program[me] and free
copies of software.

I can download free copies of RHEL 7.x, 8.x, and 9.x :) Just that I
dunno which RHEL version is better. Is RHEL 9.0 the best out of 7.x,
8.x and 9.x?

RedHat is a stable OpSys. Accordingly, it doesn't much matter which
version. The general assumption is that the more recent distribution has
more advanced facilities, eg improved security features in RHEL9.

As another post says, Fedora is closer to the bleeding-edge of Linux
development.

RHEL 9.0 is also quite close to the bleeding edge of Linux
development. It has Linux kernel version 5.14.0.


Be aware that there are many methods of adding Python. For example, if
your training is based on the Anaconda [Python] distribution, then it is
irrelevant which version of Python comes with the Linux distro. As
mentioned before, if you advance to developing in [Python] virtual
environments, then each of these could run a different version of
Python. Similarly, using a VM...

The question is relatively minor. More important to 'get going'!

I am going to get going on learning Python with RHEL 9.0.

(also mentioned previously: relatively easy to change (Python or distro)
'later'!)
--
Regards,
=dn

Regards,

Mr. Turritopsis Dohrnii Teo En Ming
Targeted Individual in Singapore
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mxIe1U15eEI7AomZUwIzjK0lVTfFXHto6Atu7jqaE58V0YUQc2K8s9LrjPDNjOXjQ5NB3Tu-cqbRDnx0pmu6$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Exclude 'None' from list comprehension of dicts

[Weatherby,Gerard]

Or:

data = [d for d in [get_job_efficiency_dict(job_id) for job_id in job_ids] if d 
is not None]

or

for job_id in job_ids:
if (d := get_job_efficiency_dict(job_id)) is not None:
  data.append(d)


Personally, I’d got with the latter in my own code.

—
Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular 
Biology and Biophysics
 UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
On Aug 4, 2022, 2:52 PM -0400, MRAB , wrote:
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2022-08-04 12:51, Loris Bennett wrote:
Hi,

I am constructing a list of dictionaries via the following list
comprehension:

data = [get_job_efficiency_dict(job_id) for job_id in job_ids]

However,

get_job_efficiency_dict(job_id)

uses 'subprocess.Popen' to run an external program and this can fail.
In this case, the dict should just be omitted from 'data'.

I can have 'get_job_efficiency_dict' return 'None' and then run

filtered_data = list(filter(None, data))

but is there a more elegant way?

I'm not sure how elegant it is, but:

data = [result for job_id in job_ids if (result :=
get_job_efficiency_dict(job_id)) is not None]
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iqxhYMoHcYQY1xohGCpafpBKZIUcGEV6Zj1-RLzOCF61TUXGr-8oh9HLuL-H8w4gxgDCypcOYOYkqNXLJxUIqhWd$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Trying to understand nested loops

[Weatherby,Gerard]

It’s also a poor code example. Doing a pointless double loop is not good 
instructional practice, especially when simpler alternatives exist. e.g.

for i in range(3):
for j in range(-2.-7,-2):
print(i +j )


—
Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular 
Biology and Biophysics
 UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
On Aug 5, 2022, 4:38 AM -0400, ojomooluwatolami...@gmail.com 
, wrote:

var = 0
for i in range(3):
for j in range(-2,-7,-2):
var += 1
print(var)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to replace an instance method?

[Weatherby,Gerard]

Just subclass and override whatever method you wish to modify
“Private” is conceptual. Mostly it means when the next version of a module 
comes out, code that you wrote that accesses *._ parts of the module might 
break.
___


import pandas


class MyClass(pandas.ExcelFile.OpenpyxlReader):

 def _convert_cell(self, cell, convert_float: bool) -> 'Scalar':
 """override"""
 # do whatever you want, or call the base class version
 return super()._convert_cell(cell, convert_float)

—
Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular 
Biology and Biophysics
 UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
On Sep 17, 2022, 5:29 PM -0400, Ralf M. , wrote:
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Am 17.09.2022 um 00:35 schrieb Dan Stromberg:


On Fri, Sep 16, 2022 at 2:06 PM Ralf M. mailto:ral...@t-online.de>> wrote:

I would like to replace a method of an instance, but don't know how to
do it properly.


You appear to have a good answer, but... are you sure this is a good idea?

It's definitely a dirty hack.

It'll probably be confusing to future maintainers of this code, and I
doubt static analyzers will like it either.

I agree that I will have to add sufficient comments for the future
maintainer, should there ever be one (and even for me to still
understand it next year). I don't use static analyzers.

I'm not the biggest fan of inheritance you'll ever meet, but maybe this
is a good place for it?

Using a derived version of the class in question to overwrite the
method was my first idea, however I don't instantiate the class in
question myself, it is instantiated during the initialisation of
another class, so I would at least have to derive a modified version of
that as well. And that code is rather complex, with metaclasses and
custom decorators, and I feel uncomfortable messing with that, while
the method I intend to change is quite simple and straightforward.

In case anybody is interested what I'm trying to achieve:

It's simple in pandas to read an excel file into a dataframe, but only
the cell value is read. Sometimes I need more / other information, e.g.
some formatting or the hyperlink in a cell. Reopening the file with
openpyxl and getting the info is possible, but cumbersome.
Looking into the pandas code for reading excel files (which uses
openpyxl internally) I noticed a method (of an internal pandas class)
that extracts the value from an openpyxl cell. This method is rather
simple and seems the ideal spot to change to get what I want.

My idea is to instantiate pandas.ExcelFile (official pandas API), get
the reader instance (an attribute of the ExcelFile object) and modify
the method of the reader instance.

The fact that the method I change and the ExcelFile attribute containing
the reader are both private (start with _) doesn't make it any better,
but I'm desperate enough to be willing to adapt my code to every major
pandas release, if necessary.

Ralf M.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mYWFkAugwhU4HgCv9nRg1vSJhyJCA8RApcnyGTRNGQYTTmvVigqANAagTbBwo96YFdHmzfCYU8gN3KpVmcrmOg$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)

[Weatherby,Gerard]

The obvious way to avoid log generation is:

if logger.isEnableFor(logging.DEBUG):
logger.debug( expensive processing )


Of course, having logging alter program flow could lead to hard to debug bugs.

From: Python-list  on 
behalf of Barry 
Date: Friday, October 7, 2022 at 1:30 PM
To: MRAB 
Cc: python-list@python.org 
Subject: Re: Ref-strings in logging messages (was: Performance issue with 
CPython 3.10 + Cython)
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

> On 7 Oct 2022, at 18:16, MRAB  wrote:
>
> On 2022-10-07 16:45, Skip Montanaro wrote:
>>> On Fri, Oct 7, 2022 at 9:42 AM Andreas Ames 
>>> wrote:
>>> 1. The culprit was me. As lazy as I am, I have used f-strings all over the
>>> place in calls to `logging.logger.debug()` and friends, evaluating all
>>> arguments regardless of whether the logger was enabled or not.
>>>
>> I thought there was some discussion about whether and how to efficiently
>> admit f-strings to the logging package. I'm guessing that's not gone
>> anywhere (yet).
> Letting you pass in a callable to call might help because that you could use 
> lambda.

Yep, that’s the obvious way to avoid expensive log data generation.
Would need logging module to support that use case.

Barry

> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$
>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)

[Weatherby,Gerard]

Logging does support passing a callable, if indirectly. It only calls __str__ 
on the object passed if debugging is enabled.

class Defer:

def __init__(self,fn):
self.fn = fn

def __str__(self):
return self.fn()

def some_expensive_function():
return "hello"

logging.basicConfig()
logging.debug(Defer(some_expensive_function))


From: Python-list  on 
behalf of Barry 
Date: Friday, October 7, 2022 at 1:30 PM
To: MRAB 
Cc: python-list@python.org 
Subject: Re: Ref-strings in logging messages (was: Performance issue with 
CPython 3.10 + Cython)
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

> On 7 Oct 2022, at 18:16, MRAB  wrote:
>
> On 2022-10-07 16:45, Skip Montanaro wrote:
>>> On Fri, Oct 7, 2022 at 9:42 AM Andreas Ames 
>>> wrote:
>>> 1. The culprit was me. As lazy as I am, I have used f-strings all over the
>>> place in calls to `logging.logger.debug()` and friends, evaluating all
>>> arguments regardless of whether the logger was enabled or not.
>>>
>> I thought there was some discussion about whether and how to efficiently
>> admit f-strings to the logging package. I'm guessing that's not gone
>> anywhere (yet).
> Letting you pass in a callable to call might help because that you could use 
> lambda.

Yep, that’s the obvious way to avoid expensive log data generation.
Would need logging module to support that use case.

Barry

> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$
>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Weatherby,Gerard]

PyCharm.

Does a good job of separating these are really errors from do you really mean 
that warnings from this word is spelled right.

https://www.jetbrains.com/pycharm/

From: Python-list  on 
behalf of Antoon Pardon 
Date: Sunday, October 9, 2022 at 6:11 AM
To: python-list@python.org 
Subject: What to use for finding as many syntax errors as possible.
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I would like a tool that tries to find as many syntax errors as possible
in a python file. I know there is the risk of false positives when a
tool tries to recover from a syntax error and proceeds but I would
prefer that over the current python strategy of quiting after the first
syntax error. I just want a tool for syntax errors. No style
enforcements. Any recommandations? -- Antoon Pardon
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kxDZilNf74VILuntVEzVZ4Wjv6RPr4JUbGpWrURDJ3CtDNAi9szBWweqrDM-uHy-o_Sncgrm2BmJIRksmxSG_LGVbBU$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Weatherby,Gerard]

try:
open(disk)
except:
error(“Can’t open disk”)
lots of things

From: Python-list  on 
behalf of Karsten Hilbert 
Date: Monday, October 10, 2022 at 5:46 AM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Am Sun, Oct 09, 2022 at 09:58:14AM + schrieb Stefan Ram:

>   I often follow this rule. For me, it's about readability. Compare:
>
> if not open( disk ):
> error( "Can't open disk" )
> else:
> printf( "now imagine there's some larger block here" )
... ad infinitum 

Should this not be

if not open( disk ):
error( "Can't open disk" )
else:
do_lots_of_things_with(disk)

as for readability ?

Or even

if not open( disk ):
error( "Can't open disk" )
return what_needs_to_be_returned

do_lots_of_things_with(disk)

The latter version may need some code reorganization, though.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nWwvwKC2eetL3YQnTS4jDih5YRZ_ziu4gPuU73R7LDX8-Kq0bXR-h4E_0kJopk-ud2oeAq6NwvykTB40o82ris6pM1aC$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Weatherby,Gerard]

Core developer Raymond Hettinger explains the history starting at 15:40 
https://www.youtube.com/watch?v=OSGv2VnC0go

(which I found on stackoverflow 
https://stackoverflow.com/questions/9979970/why-does-python-use-else-after-for-and-while-loops
 )

TL:DR
The “else” is a historical artificial from the way developers thought during 
the transition from unstructured (i.e. “GOTO”) programming to structured 
programming. Since we all do structured now, it seems odd.


From: Python-list  on 
behalf of Calvin Spealman 
Date: Monday, October 10, 2022 at 10:38 AM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On Sat, Oct 8, 2022 at 5:35 PM rbowman  wrote:

> On 10/7/22 21:32, Axy wrote:
> > So, seriously, why they needed else if the following pieces produce same
> > result? Does anyone know or remember their motivation?
>
> In real scenarios there would be more logic in the for block that would
> meet a condition and break out of the loop. If the condition is never
> met, the else block runs. To steal from w3schools:
>
>
> fruits = ["apple", "peach", "cherry"]
> for x in fruits:
>print(x)
>if x == "banana":
>  break
> else:
>print("Yes we got no bananas")
>

I wonder if for/else could have been less confusing if it was referred to
as for-break-else and if the else clause was only valid syntax if the for
loop actually contained a break statement in the first place.

--

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.speal...@redhat.com  M: +1.336.210.5107
[image: 
https://urldefense.com/v3/__https://red.ht/sig__;!!Cn_UX_p3!lSA-VzlSq_UQvMUO2npcFCxacA2jOlt2ZD6LCiwdfMGpYexlKgYpIgDCxlHPs473PNyH7cww2ufihhXbt2Z5$
  ] 

TRIED. TESTED. TRUSTED. 

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!lSA-VzlSq_UQvMUO2npcFCxacA2jOlt2ZD6LCiwdfMGpYexlKgYpIgDCxlHPs473PNyH7cww2ufihnTgBbx-$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Weatherby,Gerard]

pylint, at least, provides a warning:

fe.py:4:0: W0120: Else clause on loop without a break statement 
(useless-else-on-loop)


sum = 0
for i in range(5):
sum += i
else:
print("Always executes")
print(sum)


From: Python-list  on 
behalf of Axy via Python-list 
Date: Monday, October 10, 2022 at 1:10 PM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

> On 10/10/2022 15:52, Weatherby,Gerard wrote:
>> I wonder if for/else could have been less confusing if it was
>> referred to
>> as for-break-else and if the else clause was only valid syntax if the
>> for
>> loop actually contained a break statement in the first place.
>
> Sounds reasonable. It would be something alike UnboundLocalError when
> a local variable referenced before assignment. If they won't remove
> "else" completely in far future, that checking really worths
> implementing now.

Actually, I think a warning would be sufficient, as in the following
quick prototype. If someone can implement this quickly in CPython, that
would be great (last time I hacked it, it was 2.4)

Axy.

import ast

tree = ast.parse('''
# sample code
a = 0
for i in 'asd':
 a += i
 while x:
 pass
 else:
 print('wow')
 break
 print(i)
else:
 print(0)
''', mode='exec')

def check_ast(node):
 if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
 if node.orelse and have_no_break(node.body):
 print(f'Warning: the loop at line {node.lineno} has no
"break" statement,'
   f' "else" clause at line {node.orelse[0].lineno}
won\'t run')
 else:
 for child in ast.iter_child_nodes(node):
 check_ast(child)

def have_no_break(loop_body):
 for node in loop_body:
 if isinstance(node, (ast.For, ast.AsyncFor, ast.While)):
 # nested loop
 check_ast(node)
 elif isinstance(node, ast.Break):
 return False
 elif isinstance(node, list):
 for child in ast.iter_child_nodes(node):
 if have_no_break(child) == False:
 return False
 return True


for node in tree.body:
 check_ast(node)
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mlK4jRkfDC_akw-fIqWaMVf707GQsiyvj_sRHTsFnuG4ak5mKWwSavtz4njlBNIu1H0VHrR9gyjuQpxGqZ1dacU1Xw$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mlK4jRkfDC_akw-fIqWaMVf707GQsiyvj_sRHTsFnuG4ak5mKWwSavtz4njlBNIu1H0VHrR9gyjuQpxGqZ1dacU1Xw$>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What to use for finding as many syntax errors as possible.

[Weatherby,Gerard]

Sure it does. They’re optional and not enforced at runtime, but I find them 
useful when writing code in PyCharm:

import os
from os import DirEntry

de : DirEntry
for de in os.scandir('/tmp'):
print(de.name)

de = 7
print(de)

Predeclaring de allows me to do the tab completion thing with DirEntry fields / 
methods

From: Python-list  on 
behalf of avi.e.gr...@gmail.com 
Date: Monday, October 10, 2022 at 10:11 PM
To: python-list@python.org 
Subject: RE: What to use for finding as many syntax errors as possible.
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Michael,

A reasonable question. Python lets you initialize variables but has no
explicit declarations.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Weatherby,Gerard]

As did I.

tree = ET.parse(lfile)
for child in tree.getroot():
if child.tag == 'server':
break
else:
raise ValueError(f"server tag not found in {lfile}")

I think there are other places I could be using it, but honestly I tend to 
forget it’s available.

From: Python-list  on 
behalf of Stefan Ram 
Date: Wednesday, October 12, 2022 at 2:22 PM
To: python-list@python.org 
Subject: Re: for -- else: what was the motivation?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Axy  writes:
>So, seriously, why they needed else if the following pieces produce same
>result? Does anyone know or remember their motivation?

  Just wrote code wherein I used "else"! This:

import locale
for name in( 'de', 'de_DE', 'deu_deu', 'deu', 'German', 'Deutsch' ):
try: locale.setlocale( locale.LC_ALL, name ); break
except locale.Error: pass
else: print( "Programm kann deutsche Schreibweise nicht einrichten." )

  .


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iyDac-XjNlj78G0XwNzZ-FEHyuCZIy33n3cI9MUDM_FnEdR04mSQ5Ln0OA1ETUNloyH24iY9meNHVdixLgWRYL8$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Find the path of a shell command

[Weatherby,Gerard]

Some thoughts / notes:
1. On our Ubuntu-Server 20.04.3 based systems, /bin/rm and /usr/bin/rm are hard 
links to the same file.

2. Put the following in a bash script and run it from cron. Then you can see 
what your environment is. Mine has PATH=/usr/bin:/bin in it.

#!/bin/bash
env > /tmp/env$$

3. You can background Python just as well as you can a separate rm invocation.
if os.fork() == 0:
os.unlink(….)








From: Python-list  on 
behalf of Grant Edwards 
Date: Wednesday, October 12, 2022 at 5:03 PM
To: python-list@python.org 
Subject: Re: Find the path of a shell command
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2022-10-12, 2qdxy4rzwzuui...@potatochowder.com 
<2qdxy4rzwzuui...@potatochowder.com> wrote:
> On 2022-10-12 at 17:43:18 +0100, Paulo da Silva 
>  wrote:
>>
>> > Probably you could use os.unlink[1] with no problem.
>>
>> No, because I need to launch several rm's that keep running after the script
>> ends.
>
> rm doesn't take that long.

Recursive ones can take a long time.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ny8-fSGJEE3wk-Yq188xuT5aamgIUVI6SJQHkNaxMp11JelflHEJ2SQcYqqSC1s8pxkifVVdFyeEcbSv1ZJlqoaxuG_m$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Find the path of a shell command (shutil.which)

[Weatherby,Gerard]

Not going to do any good if it’s not on the PATH in the first place:

https://docs.python.org/3/library/shutil.html

shutil.which(cmd, mode=os.F_OK | os.X_OK, path=None)
Return the path to an executable which would be run if the given cmd was 
called. If no cmd would be called, return None.
mode is a permission mask passed to 
os.access(), by default 
determining if the file exists and executable.
When no path is specified, the results of 
os.environ() are used, 
returning either the “PATH” value or a fallback of 
os.defpath.


From: Python-list  on 
behalf of Thomas Passin 
Date: Wednesday, October 12, 2022 at 5:39 PM
To: python-list@python.org 
Subject: Re: Find the path of a shell command
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 10/12/2022 12:00 AM, Paulo da Silva wrote:
> Hi!
>
> The simple question: How do I find the full path of a shell command
> (linux), i.e. how do I obtain the corresponding of, for example,
> "type rm" in command line?
>
> The reason:
> I have python program that launches a detached rm. It works pretty well
> until it is invoked by cron! I suspect that for cron we need to specify
> the full path.
> Of course I can hardcode /usr/bin/rm. But, is rm always in /usr/bin?
> What about other commands?
>
> Thanks for any comments/responses.
> Paulo
>

Python has a command that usually does the job:

import shutil, os
executable_path = shutil.which(exename, os.X_OK)

If you know that the executable you want is on some non-standard path,
you can or them together:

executable_path = shutil.which(exename, os.X_OK)\
or shutil.which(exename, os.X_OK, other_path)

Presumably the shutil command uses which behind the scenes, but that
doesn't matter.

Now you can include this location when you run the executable from
Python.  If you need to run a system command from a batch file and not
from Python, you will either need to have the right path in effect when
the batch file is run, or manually include the full path to the file in
the batch file.

BTW, on Linux Mint, which is derived from Ubuntu and Debian, rm is at

$ which rm
/bin/rm



--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!lyWawGnR8ddR5pCSS5bC09WInoHss_eleSRXxsjbIL_ndHz2TPW246oagSmaBVekZyVjzO7zKsGqN9NpLtM$
-- 
https://mail.python.org/mailman/listinfo/python-list

systemd Re: Find the path of a shell command

[Weatherby,Gerard]

Only the first one or two require more work. The rest are your copy-pastes from 
the last time you did one.

Typically I only have to changed descriptions, the executable in the *service 
and the timing in the *timer. They’re lot more readable and flexible than the 
cronjobs.

From: Python-list  on 
behalf of Albert-Jan Roskam 
Date: Friday, October 14, 2022 at 1:59 PM
To: Peter J. Holzer 
Cc: python-list@python.org 



   =
   Lately I've been using systemd timers instead of cronjobs. They are easier
   to debug (journalctl) but require a bit more work to write. Systemd is
   available on Fedora & friends and Debian based systems, maybe more. It has
   no builtin MAILTO. I use an OnFailure stanza to send a Slack message with
   curl instead.
   
https://urldefense.com/v3/__https://www.freedesktop.org/software/systemd/man/systemd.timer.html__;!!Cn_UX_p3!l6ZRc2Ur6rwspaC1V71wsENqmq5RNIR3lU2suC1LJLzYp097e-r2NwNBD3a_RhjuUkrSmfq3emmUsY-1m0PxT2TM$
   
https://urldefense.com/v3/__https://unix.stackexchange.com/questions/278564/cron-vs-systemd-timers__;!!Cn_UX_p3!l6ZRc2Ur6rwspaC1V71wsENqmq5RNIR3lU2suC1LJLzYp097e-r2NwNBD3a_RhjuUkrSmfq3emmUsY-1m6sDQHR0$
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!l6ZRc2Ur6rwspaC1V71wsENqmq5RNIR3lU2suC1LJLzYp097e-r2NwNBD3a_RhjuUkrSmfq3emmUsY-1m3Q-bOmf$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Operator: inappropriate wording?

[Weatherby,Gerard]

No. If the docs say in one place a comma is not an operator, they shouldn’t 
call it an operator in another place.

I’ve submitted a pull request https://github.com/python/cpython/pull/98736 -- 
we’ll have to see what The Powers That Be think.

From: Python-list  on 
behalf of elas tica 
Date: Wednesday, October 26, 2022 at 3:32 PM
To: python-list@python.org 
Subject: Operator: inappropriate wording?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Quotes from The Python Language Reference, Release 3.10.8:

- Note that tuples are not formed by the parentheses, but rather by use of the 
comma operator (p. 66)
- Note: If the object is a class instance and the attribute reference occurs on 
both sides of the assignment operator (p. 86)
- The second half of the list, the augmented assignment operators, serve 
lexically as delimiters, but also perform an operation (p. 15)



Do you agree with this use of the term "operator"?

Because there is no such "comma operator" in Python as explained by the 
official FAQ: 
https://urldefense.com/v3/__https://docs.python.org/3/faq/programming.html*what-s-up-with-the-comma-operator-s-precedence__;Iw!!Cn_UX_p3!nL9TZBaXZH87JOWNN7D_3E0ytfdXH8tkUxvTdF4dXbVqOJS0yF0C7idVg4ZGee7FiFyPQqqNRcNeVeWVu5edfg$

And, =, += and the like are not operators since (a=b), (a+=b), etc have no 
value. There is no assignment operator instead there exists an assignment 
statement. The only assignment operator I can figure out is the walrus 
operator. To confirm, The Python Language Reference gives here:

https://urldefense.com/v3/__https://docs.python.org/3/reference/lexical_analysis.html*operators__;Iw!!Cn_UX_p3!nL9TZBaXZH87JOWNN7D_3E0ytfdXH8tkUxvTdF4dXbVqOJS0yF0C7idVg4ZGee7FiFyPQqqNRcNeVeVnKYO3BA$

the set of tokens considered as operator and the =, += tokens are not listed 
whereas := is.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nL9TZBaXZH87JOWNN7D_3E0ytfdXH8tkUxvTdF4dXbVqOJS0yF0C7idVg4ZGee7FiFyPQqqNRcNeVeW3iNHD7g$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: an oop question

[Weatherby,Gerard]

I don’t understand your implementation enough to comment specifically. What’s 
the definition of Pair? (i.e. what methods and public attributes does it have?) 
(I manage to escape Lisp as an undergrad)

To answer your question generally, Union’s are not OO. If you want Pair and 
Stack to have the same interface create an abstract base class and derive both 
of them from it.

https://docs.python.org/3/library/abc.html


From: Python-list  on 
behalf of Julieta Shem 
Date: Sunday, October 30, 2022 at 5:51 PM
To: python-list@python.org 
Subject: an oop question
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I have a question about a particular case I'm working on.  I'm studying
OOP.  To ask the question, I'm going to have to introduce you my context
here, so you'll need to bear with me.  If you'd like to see the question
right away, go to the section ``My difficulty in encapsulating a
union''.

(*) Introduction

I wrote the classes

  class Empty:
...
  class Pair:
...

With them, I can (sort of) make a union of them and build a Lisp-like
list by defining that my Lisp-like list is either Empty or a Pair.  For
instance, here's a Lisp-like sequence that represents the string "abc".

>>> Pair.fromIterable("abc")
Pair('a', Pair('b', Pair('c', Empty(

So far so good.  (``Full'' code at the end of this message, if you'd
like to more carefully look into my reasoning there.)

(*) How to build a stack?

These Lisp-like sequences are clearly a stack.  You pop an element from
it by grabbing the first element.  You push an element on to it by just
pairing the new element with the current stack.  For instance, let's pop
the 1-string "a" off of the stack.

>>> ls = Pair.fromIterable("abc")
>>> ls.first
'a'
>>> ls = ls.rest
>>> ls
Pair('b', Pair('c', Empty()))

Done.  Let's push it back on.

>>> ls = Pair("a", ls)
>>> ls
Pair('a', Pair('b', Pair('c', Empty(

So far so good, but when it comes to building a better user interface
for it I have no idea how to do it.  I think one of the purposes of OOP
is to organize code hierarchically so that we can reuse what we wrote.
So I tried to make a Stack by inheriting Pair.

class Stack(Pair):
pass

>>> Stack(1, Empty())
Stack(1, Empty())

Then I wrote pop and push.

>>> Stack(1, Empty()).pop()
1

>>> Stack(1, Empty()).push(2)
Stack(2, Stack(1, Empty()))

So far so good.  Now let me show you what I can't do.

(*) The difficulty of encapsulating a union

The Lisp-like sequences we're building here are union-like data
structures.  A /sequence/ is either Empty() or Pair(..., /sequence/).  I
have not found a way to represent this either-or datastructure with a
class.  For example, there is no way right now to build an empty Stack
by invoking the Stack constructor.

>>> Stack()
Traceback (most recent call last):
  File "", line 1, in 
TypeError: Pair.__init__() missing 2 required positional arguments: 'first' and 
'rest'

As I designed, an empty Stack is represented by Empty(), which is a
whole other object.  Users will have to know this.  I wish I did not
have to burden my users with such knowledge.  For instance, I wish the
answer was "Stack()" to the question of -- ``how can I build an empty
stack?''

My desire seems to imply that I need a union-like data structure.  If
Stack is invoked with no arguments, it should produce Empty(), otherwise
it produces Pair() as it does today.

How can I achieve that?

(*) Code

class Pair:
  def __init__(self, first, rest):
if not isinstance(rest, Pair) and not isinstance(rest, Empty):
  raise ValueError("rest must be Empty or Pair")
self.first = first
self.rest = rest
  def fromIterable(it):
if len(it) == 0:
  return Empty()
else:
  return Pair(it[0], Pair.fromIterable(it[1:]))
  def __str__(self):
return "{}({!r}, {})".format(self.__class__.__name__, self.first, 
str(self.rest))
  def __repr__(self):
return str(self)
  def __len__(self):
return 1 + self.rest.__len__()

class Empty:
  def __len__(self):
return 0
  def __str__(self):
return  "Empty()"
  def __repr__(self):
return self.__str__()
  def __new__(clss):
if not hasattr(clss, "saved"):
  clss.saved = super().__new__(clss)
return clss.saved

class Stack(Pair):
  def pop(self):
return self.first
  def push(self, x):
return Stack(x, self)
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iK-T12707FwUmb1Qqxlxawj0i9z1tR9aCPWPoHzh5veCzlaFBY9pKfFZwGqg3sDjUqLPC2IZgGw4QI3sRg$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: an oop question

[Weatherby,Gerard]

I think:

class Stack:
def __init__( self, *args ):
self.data = args

def __str__( self ):
return f"Stack({','.join(str(x) for x in self.data)})"

gives equivalent output for the if len(args) is 0 or 2, if it’s okay for 
self.data to be a tuple.

class Stack:
def __init__( self, *args ):
self.data = list(args)

def __str__( self ):
return f"Stack({','.join(str(x) for x in self.data)})"

If it’s desired self.data be a list.


From: Python-list  on 
behalf of Stefan Ram 
Date: Tuesday, November 1, 2022 at 3:43 PM
To: python-list@python.org 
Subject: Re: an oop question
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Julieta Shem  writes:
>clarify.  If I wish for an empty stack, I wish I could just say
 Stack()
>Stack()
>and if I wish for a nonempty stack, I'd write
 Stack(1, Stack(2, Stack(3, Stack(
>Stack(1, Stack(2, Stack(3, Stack(

  If this is all,

  main.py

class Stack:
def __init__( self, *args ):
self.data = [ args[ 0 ], args[ 1 ]]if len( args ) else []
def __str__( self ):
if len( self.data ):
return f"Stack({self.data[0]}, {self.data[1]})"
else:
return f"Stack()"

print( Stack() )

print( Stack(1, Stack(2, Stack(3, Stack( )

  output

Stack()
Stack(1, Stack(2, Stack(3, Stack(

  .


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!itEYnwU5jJ0z8_rkW_q_ogw3ZJUNdHdMNkMLpSAqBdozBNrr7NqPs_gNsbx8W9uXRLZpG38C9an17Yx2zUf-mSA$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: an oop question

[Weatherby,Gerard]

C++/Java class variables can be public, protected (accessible to class and 
subclasses) or private (accessible only to class). Of course the language 
protections can be hacked around.

Python does conceptual private variables by using the single underscore: 
object._myvar is considered private

From: Python-list  on 
behalf of Alan Gauld 
Date: Thursday, November 3, 2022 at 6:45 AM
To: Julieta Shem , python-list@python.org 

Subject: Re: an oop question
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 03/11/2022 00:25, Julieta Shem wrote:

>> |OOP to me means only messaging, local retention and protection and
>> |hiding of state-process, and extreme late-binding of all things.
>
> I'm wondering how Python fails to satisfy his definition.

Python doesn't do any form of data protection/hiding. All
attributes are public by default.

In Smalltalk all attributes are private, with no way to
make them public... Actually in C++/Java terms I believe
they are "protected" because subclasses can access
them(I think?).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: an oop question

[Weatherby,Gerard]

https://www.oreilly.com/library/view/beginning-c-30/9780470261293/9780470261293_a_short_history_of_object-oriented_progr.html
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to manage python shebang on mixed systems?

[Weatherby,Gerard]

A possible solution is here.

https://superuser.com/questions/815323/can-i-have-a-conditional-shebang

From: Python-list  on 
behalf of jak 
Sent: Sunday, November 6, 2022 2:51:10 PM
To: python-list@python.org 
Subject: Re: How to manage python shebang on mixed systems?

*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Il 06/11/2022 11:03, Chris Green ha scritto:
> I have a number of python scripts that I run on a mix of systems.  I
> have updated them all to run on python 3 but many will also run quite
> happily with python 2.  They all have a #!/usr/bin/python3 shebang.
>
> This works almost everywhere but there is one system where only
> python 2 is available (at /usr/bin/python).
>
> I don't have python 2 on any of the systems I manage myself now so a
> #!/usr/bin/python shebang will fail.
>
> Is there a neat way of handling this?  I could write a sort of wrapper
> script to run via the shebang but that seems overkill to me.
>

hi,
If you can call Python from the shell prompt, then you could remove the
path from shebang:

#!python

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ke_gwJrnkkPx4fdk8CkLm6Qd2lmYIl7st4qz7Mmn0G8BerBOEwRWBfm51eFZ-Ut4WCTXTGoUEP5MTWYjmZE$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to manage python shebang on mixed systems?

[Weatherby,Gerard]

Write the wrapper script.

#!/bin/bash
if [ $(hostname) == "oldystem" ]; then
exec /usr/bin/python $*
else
exec /usr/bin/python2 $*
fi

From: Python-list  on 
behalf of Chris Green 
Date: Sunday, November 6, 2022 at 3:22 PM
To: python-list@python.org 
Subject: Re: How to manage python shebang on mixed systems?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

rbowman  wrote:
> On Sun, 6 Nov 2022 10:03:50 +, Chris Green wrote:
>
>
> > Is there a neat way of handling this?  I could write a sort of wrapper
> > script to run via the shebang but that seems overkill to me.
>
> Can you symlink?

Not really, since the system where there's no python3 is one where I
only have user access.

--
Chris Green
·
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gN4FcCAwYqK8bafmF2dr_EB9nahZcdNmU91bHuymdlakU27cNTnIwQ_FckF9PBZlllGnW_vlEjQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Argument name should be lowercase

[Weatherby,Gerard]

PEP 8 doesn’t explicitly list a naming convention for function parameters, but 
every example shows them as lowercase, even though the function doesn’t modify 
them.

See also the Python tutorial ( 
https://docs.python.org/3/tutorial/controlflow.html#defining-functions ), which 
also shows all parameters as lowercase.

I’d personally find it weird to see an all-cap parameter (Why are you 
yelling?). I expect ALL_CAPS to be hardcoded values.



From: Python-list  on 
behalf of dn 
Date: Friday, November 11, 2022 at 3:56 AM
To: 'Python' 
Subject: Argument name should be lowercase
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

PyCharm is warning against using an identifier of all upper-case letters
as a function's parameter, saying "Argument name should be lowercase".
(weak, code smell)


The application consists of three+ files:
- configuration
- mainline script
- module of workbook functions

The mainline reads the configuration parameters to set the application's
environment. All of the config/settings are constants. Some of them
appear as a dictionary (PRICES_WORKBOOK) defining a workbook's
(spreadsheet's) parameters, eg the filename, which work-sheet to use, etc.


The mainline calls the relevant functions from within the module,
as-needed:-

import prices_workbook as pw
...
product_prices = pw.build_product_prices_dictionary( PRICES_WORKBOOK )


The module's function definition is:

def build_product_prices_dictionary( WORKBOOK_DEFINITIONS:dict )->dict:
...
 price_array = xl.iget_array(
 file_name=WORKBOOK_DEFINITIONS[ "file_name" ],
 ...

(the function def is flagged, as above)


A quick scan of PEP-008 failed to yield anything relevant. Why is this
frowned upon as poor Python, or a matter of style?

Yes, a dict is mutable, but the upper-case denoting a constant indicates
that none of its values are to be changed by the programmer.

As far as the function is concerned, the dict and its contents are
constants.
(but the dict can't be treated as a global ENV[IRONMENT] object, because
it has to cross into the module's namespace)


Is passing the dict as an argument/parameter considered to be
incompatible with its designation as a constant?

Perhaps the style should be more enum-like, ie the dict's name in
lower-case, with the key-named in upper case, eg

 workbook_definitions[ "FILE_NAME" ]


Am not particularly concerned by the IDE raising this as a 'problem' -
will quite happily ignore and carry-on; but am curious as to the logic
behind the analysis - and why it doesn't come readily to mind.

Advice, comments, critique welcome!

--
Regards,
=dn
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!h56Cia7ERYDmxaCnEo0k9hfXz-mTJrz43UqHjbfhwLjutjhQE1QU975lUXTBf38la5kXAkBHdzyzOY4XAObbAPOa-ebC-HNY$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Argument name should be lowercase

[Weatherby,Gerard]

If you wanted to document/enforce the input argument is immutable, you could do 
one of the following. You’d get a type warning in PyCharm with bad_func or 
equivalent, and bad_func2 will fail at runtime.

def bad_func(x: typing.Mapping):
"""Get type warning if x modified"""
x[3] = 7


def bad_func2(x: types.MappingProxyType):
"""Get runtime error if x modified"""
if not isinstance(x, types.MappingProxyType):
x = types.MappingProxyType(x)
x[3] = 8


From: Python-list  on 
behalf of dn 
Date: Friday, November 11, 2022 at 3:56 AM
To: 'Python' 
Subject: Argument name should be lowercase
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

PyCharm is warning against using an identifier of all upper-case letters
as a function's parameter, saying "Argument name should be lowercase".
(weak, code smell)


The application consists of three+ files:
- configuration
- mainline script
- module of workbook functions

The mainline reads the configuration parameters to set the application's
environment. All of the config/settings are constants. Some of them
appear as a dictionary (PRICES_WORKBOOK) defining a workbook's
(spreadsheet's) parameters, eg the filename, which work-sheet to use, etc.


The mainline calls the relevant functions from within the module,
as-needed:-

import prices_workbook as pw
...
product_prices = pw.build_product_prices_dictionary( PRICES_WORKBOOK )


The module's function definition is:

def build_product_prices_dictionary( WORKBOOK_DEFINITIONS:dict )->dict:
...
 price_array = xl.iget_array(
 file_name=WORKBOOK_DEFINITIONS[ "file_name" ],
 ...

(the function def is flagged, as above)


A quick scan of PEP-008 failed to yield anything relevant. Why is this
frowned upon as poor Python, or a matter of style?

Yes, a dict is mutable, but the upper-case denoting a constant indicates
that none of its values are to be changed by the programmer.

As far as the function is concerned, the dict and its contents are
constants.
(but the dict can't be treated as a global ENV[IRONMENT] object, because
it has to cross into the module's namespace)


Is passing the dict as an argument/parameter considered to be
incompatible with its designation as a constant?

Perhaps the style should be more enum-like, ie the dict's name in
lower-case, with the key-named in upper case, eg

 workbook_definitions[ "FILE_NAME" ]


Am not particularly concerned by the IDE raising this as a 'problem' -
will quite happily ignore and carry-on; but am curious as to the logic
behind the analysis - and why it doesn't come readily to mind.

Advice, comments, critique welcome!

--
Regards,
=dn
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!h56Cia7ERYDmxaCnEo0k9hfXz-mTJrz43UqHjbfhwLjutjhQE1QU975lUXTBf38la5kXAkBHdzyzOY4XAObbAPOa-ebC-HNY$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Need max values in list of tuples, based on position

[Weatherby,Gerard]

Types are available if you want to use them. 
https://www.pythontutorial.net/python-basics/python-type-hints/

From: Python-list  on 
behalf of Pancho via Python-list 
Date: Friday, November 11, 2022 at 6:28 PM
To: python-list@python.org 
Subject: Re: Need max values in list of tuples, based on position

That was one of the things I didn't like about Python. Lack of types and
subsequent loss of intellisense is the thing I find hardest to deal with.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jAUMq5gqI7J7UJIYw7lfpUMAy-BZPyyY8uWLJLVdVLWCaVMSBzVE6uX4RWQ-YZ0c3gvnIYn1u0r0BOmdCvEVAhyK6g$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Argument name should be lowercase

[Weatherby,Gerard]

RUN is a global constant, and the method is documented, explaining why the 
parameter is there:

# Copy globals as function locals to make sure that they are available
# during Python shutdown when the Pool is destroyed.
def __del__(self, _warn=warnings.warn, RUN=RUN):

From: Python-list  on 
behalf of Stefan Ram 
Date: Friday, November 11, 2022 at 2:42 PM
To: python-list@python.org 
Subject: Re: Argument name should be lowercase
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

"Weatherby,Gerard"  writes:
>I'd personally find it weird to see an all-cap parameter

  In the source code of the standard library, all-caps
  notation is used for a parameter name sometimes
  if that parameter name is "URL" or sometimes
  when it is being initialized from an all-caps name as in:

|def __del__(self, _warn=warnings.warn, RUN=RUN):
|if self._state == RUN:
...
from "pool.py".

  ("RUN=RUN" makes RUN have the value "RUN" had
  at the time of the definition of the function.)


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kI1c1EIpXCtnz0SBHvQmJokWXPBo0rCY9MQ-IwXOZU4RyHqqKho9wMu6Ekj6GMXA0EbReT8_4GOxvoldiOsL3W8$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kI1c1EIpXCtnz0SBHvQmJokWXPBo0rCY9MQ-IwXOZU4RyHqqKho9wMu6Ekj6GMXA0EbReT8_4GOxvoldiOsL3W8$>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dealing with non-callable classmethod objects

[Weatherby,Gerard]

Use the inspect module as Cameron suggested.


import inspect


def analyze_class(clzz):
"""Analyze a class proof of concept"""
assert inspect.isclass(clzz)
for k, v in inspect.getmembers(clzz, lambda v: inspect.isfunction(v) or 
inspect.ismethod(v)):
if inspect.ismethod(v):
print(f"{v.__qualname__} -> class method ")
if inspect.isfunction(v):
sig = inspect.signature(v)
names = [n for n, _ in sig.parameters.items()]
if len(names) > 0 and str(names[0]) == 'self':
print(f"{v.__qualname__}->  probably a bound method ")
else:
print(f"{v.__qualname__}->  probably a static method ")


class Demo:

@classmethod
def us(cls):
print(cls.__name__)

@staticmethod
def double(x):
return x + x

def triple(self, y):
return 3 * y


analyze_class(Demo)

output:

Demo.double->  probably a static method
Demo.triple->  probably a bound method
Demo.us -> class method



From: Python-list  on 
behalf of Ian Pilcher 
Date: Saturday, November 12, 2022 at 11:36 AM
To: python-list@python.org 
Subject: Re: Dealing with non-callable classmethod objects
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 11/11/22 16:47, Cameron Simpson wrote:
> On 11Nov2022 15:29, Ian Pilcher  wrote:
>> * Can I improve the 'if callable(factory):' test above?  This treats
>>  all non-callable objects as classmethods, which is obviously not
>>  correct.  Ideally, I would check specifically for a classmethod, but
>>  there doesn't seem to be any literal against which I could check the
>>  factory's type.
>
> Yeah, it does feel a bit touchy feely.
>
> You could see if the `inspect` module tells you more precise things
> about the `factory`.
>
> The other suggestion I have is to put the method name in `_attrs`; if
> that's a `str` you could special case it as a well known type for the
> factory and look it up with `getattr(cls,factory)`.

So I've done this.

 class _HasUnboundClassMethod(object):
 @classmethod
 def _classmethod(cls):
 pass  # pragma: no cover
 _methods = [ _classmethod ]

 _ClassMethodType = type(_HasUnboundClassMethod._methods[0])

Which allows me to do this:

 def __init__(self, d):
 for attr, factory in self._attrs.items():
 if callable(factory):
 value = factory(d[attr])
 else:
 assert type(factory) is self._ClassMethodType
 value = factory.__func__(type(self), d[attr])
 setattr(self, attr, value)

It's a bit cleaner, although I'm not thrilled about having a throwaway
class, just to define a literal that ought to be provided by the
runtime.

--

Google  Where SkyNet meets Idiocracy


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nx6jxVGHt4Gj1WplLAV4uuhaMyS7Ry0qTCGvZm7jLCj9GbK4vto49sfmP12TTgcAT6Akjz5hJWw9JoylO_FrgQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Persisting functions typed into the shell

[Weatherby,Gerard]

Sounds like Jupyter Notebooks: https://jupyter.org


From: Python-list  on 
behalf of Stefan Ram 
Date: Saturday, November 12, 2022 at 1:48 PM
To: python-list@python.org 
Subject: Persisting functions typed into the shell
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

  Many readers here know interactive Python sessions with
  prompts like ">>>". But a "session" could be something else.
  One could imagine that when starting a new session, one
  still sees all the variables and constants defined in
  preceding sessions.

  I have implemented something like a "restore()" and a "save()"
  call. "restore()" will restore the names from the last "save()".
  "save()" will look for user-defined names (it excludes certain
  standard names and certain other names from my software) and
  save them using the "shelve" package from the standard library.

  I you know "shelve" or have read the subject line, you can
  guess what comes now:

  I cannot save user-defined functions this way!

  When a user types into the console:

|>>> def f():
|...print( "example" )
|...

  he gives source code to shell and hopes that the shell will
  cherish the memory of that function. But instead it acts as
  if from now on it does not know the source code of "f"!

  So, there seems to be no way now to persist this function
  to a file? as if by "save( f )" or something similar?
  If not the source code, then maybe some other form?

  So much for the topic of "In Python, /everything/ is an
  object"! There seem to be first and second-class objects:
  Shelveable and non-shelveable objects.


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kVssvGre00pk5iMVIWUbuGXUwcZ8veRBEuSiX-VLlkRVlJoQ96fl6CZG9zQ72Hky8qqofZhhhA5qZpkneyEz4-4$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: In code, list.clear doesn't throw error - it's just ignored

[Weatherby,Gerard]

The terminal told you what x.clear was.

The Python documentation tells you how to call it: 
https://docs.python.org/3/tutorial/datastructures.html

list.clear()
Remove all items from the list. Equivalent to del a[:].

An IDE (e.g. PyCharm) will try to autocomplete the parentheses and warn you if 
you don’t:

“Statement seems to have no effect and can be replaced with a function call to 
have effect”

From: Python-list  on 
behalf of DFS 
Date: Sunday, November 13, 2022 at 7:46 PM
To: python-list@python.org 
Subject: In code, list.clear doesn't throw error - it's just ignored
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

In code, list.clear is just ignored.
At the terminal, list.clear shows



in code:
x = [1,2,3]
x.clear
print(len(x))
3

at terminal:
x = [1,2,3]
x.clear

print(len(x))
3


Caused me an hour of frustration before I noticed list.clear() was what
I needed.

x = [1,2,3]
x.clear()
print(len(x))
0

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jRrjG0fRo46VyY0jLfD1Z5C6tXDiphZy8zi2AqN_N5BB1_OwBe_wxmsBWmIVOFQDdZnvbRq1JNeGnPg$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Are these good ideas?

[Weatherby,Gerard]

Issue 1.

Depends very much on your operating system and application environment.

Issue 2.
I usually make myself a data class to pass around. Then when I figure out I 
forgot something, I just update the dataclass and the creator and consumer of 
it.

@dataclass
class CallParameter:
first : int
second: str

etc. I use context specific names, not “first” et. al.

From: Python-list  on 
behalf of Stephen Tucker 
Date: Monday, November 14, 2022 at 12:16 PM
To: Python 
Subject: Are these good ideas?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hi,

I have two related issues I'd like comments on.

Issue 1 - Global Values

Some years ago, I had a situation where
(a) I could supply low-level functions that carry out tasks,
(b) I needed those functions to communicate with each other, but
(c) I had no access to the module that invoked my functions.

In order to achieve this, I hit on the idea that I also supply a module
that I describe as a "global values" module. This module …
(i) … defines the values that the functions use to communicate with each
other;
(ii) … is the subject of an import statement in each of my functions;
(iii) … initialises its values at the start of each run (since the import
only carries out an actual import once per session);
(iv) … acts as a repository for the values thereafter.

This solution works well.

Given that I am not particularly concerned about efficiency,
(1) Is this a reasonable way to achieve this goal?
(2) Do you know of any better ways?

Issue 2 - Passed Parameters

I am now facing another situation where I am wanting to pass 6 or 7
parameters down through several layers of logic (function A calling
function B calling ... ) and for results to be passed back.

Having had the idea described above, I am considering using it again to
save all the parameter-and-results passing.

I see nothing wrong with doing that, but I may well be missing something!

Comments, please!

Stephen Tucker.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hb_4MXkjG1NsyGoNsLJNDTOruPzfPWJKD-6vj0_2N1yqqtvz8oDZH3cT0EVkFbPTzSC19cAOgXlQdkDp7FZjwbyOjw$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Preprocessing not quite fixed-width file before parsing

[Weatherby,Gerard]

Oops. Forgot to Reformat file before sending. Here’s the proper PEP-8 (at least 
according to PyCharm)

import pandas
import logging


class Wrapper:
"""Wrap file to fix up data"""

def __init__(self, filename):
self.filename = filename

def __enter__(self):
self.fh = open(self.filename, 'r')
return self

def __exit__(self, exc_type, exc_val, exc_tb):
self.fh.close()

def __iter__(self):
"""This is required by pandas for some reason, even though it doesn't 
seem to be called"""
raise ValueError("Unsupported operation")

def read(self, n: int):
"""Read data. Replace 'grace' before | if it has underscores in it"""
try:
data = self.fh.readline()
ht = data.split('|', maxsplit=2)
if len(ht) == 2:
head, tail = ht
hparts = head.split(maxsplit=7)
assert len(hparts) == 8
if ' ' in hparts[7].strip():
hparts[7] = hparts[7].strip().replace(' ', '_')
fixed_data = f"{' '.join(hparts)} | {tail}"
return fixed_data

return data
except:
logging.exception("read")


logging.basicConfig()
with Wrapper('data.txt') as f:
df = pandas.read_csv(f, delimiter=r"\s+")
print(df)


From: Weatherby,Gerard 
Date: Wednesday, November 23, 2022 at 3:38 PM
To: Loris Bennett , python-list@python.org 

Subject: Re: Preprocessing not quite fixed-width file before parsing

This seems to work. I’m inferring the | is present in each line that needs to 
be fixed.

import pandas
import logging


class Wrapper:
"""Wrap file to fix up data"""

def __init__(self, filename):
self.filename = filename

def __enter__(self):
self.fh = open(self.filename,'r')
return self

def __exit__(self, exc_type, exc_val, exc_tb):
self.fh.close()

def __iter__(self):
"""This is required by pandas for some reason, even though it doesn't 
seem to be called"""
raise ValueError("Unsupported operation")

def read(self, n: int):
"""Read data. Replace 'grace' before | if it has underscores in it"""
try:
data = self.fh.readline()
ht = data.split('|', maxsplit=2)
if len(ht) == 2:
head,tail = ht
hparts = head.split(maxsplit=7)
assert len(hparts) == 8
if ' ' in hparts[7].strip():
hparts[7] = hparts[7].strip().replace(' ','_')
fixed_data = f"{' '.join(hparts)} | {tail}"
return fixed_data

return data
except:
logging.exception("read")

logging.basicConfig()
with Wrapper('data.txt') as f:
df = pandas.read_csv(f, delimiter=r"\s+")
print(df)


From: Python-list  on 
behalf of Loris Bennett 
Date: Wednesday, November 23, 2022 at 2:00 PM
To: python-list@python.org 
Subject: Preprocessing not quite fixed-width file before parsing
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hi,

I am using pandas to parse a file with the following structure:

Name   filesettype KB  quota  limit   in_doubt
grace |files   quotalimit in_doubtgrace
shortname  sharedhome USR14097664  524288000  545259520  0 
none |   107110   000 none
gracedays  sharedhome USR   774858944  524288000  775946240  0   5 
days |  1115717   000 none
nametoolong sharedhome USR27418496  524288000  545259520  0 
none |11581   000 none

I was initially able to use

  df = pandas.read_csv(file_name, delimiter=r"\s+")

because all the values for 'grace' were 'none'.  Now, however,
non-"none" values have appeared and this fails.

I can't use

  pandas.read_fwf

even with an explicit colspec, because the names in the first column
which are too long for the column will displace the rest of the data to
the right.

The report which produces the file could in fact also generate a
properly delimited CSV file, but I have a lot of historical data in the
readable but poorly parsable format above that I need to deal with.

If I were doing something similar in the shell, I would just pipe the
file through sed or something to replace '5 days' with, say &#

Re: Preprocessing not quite fixed-width file before parsing

[Weatherby,Gerard]

This seems to work. I’m inferring the | is present in each line that needs to 
be fixed.

import pandas
import logging


class Wrapper:
"""Wrap file to fix up data"""

def __init__(self, filename):
self.filename = filename

def __enter__(self):
self.fh = open(self.filename,'r')
return self

def __exit__(self, exc_type, exc_val, exc_tb):
self.fh.close()

def __iter__(self):
"""This is required by pandas for some reason, even though it doesn't 
seem to be called"""
raise ValueError("Unsupported operation")

def read(self, n: int):
"""Read data. Replace 'grace' before | if it has underscores in it"""
try:
data = self.fh.readline()
ht = data.split('|', maxsplit=2)
if len(ht) == 2:
head,tail = ht
hparts = head.split(maxsplit=7)
assert len(hparts) == 8
if ' ' in hparts[7].strip():
hparts[7] = hparts[7].strip().replace(' ','_')
fixed_data = f"{' '.join(hparts)} | {tail}"
return fixed_data

return data
except:
logging.exception("read")

logging.basicConfig()
with Wrapper('data.txt') as f:
df = pandas.read_csv(f, delimiter=r"\s+")
print(df)


From: Python-list  on 
behalf of Loris Bennett 
Date: Wednesday, November 23, 2022 at 2:00 PM
To: python-list@python.org 
Subject: Preprocessing not quite fixed-width file before parsing
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hi,

I am using pandas to parse a file with the following structure:

Name   filesettype KB  quota  limit   in_doubt
grace |files   quotalimit in_doubtgrace
shortname  sharedhome USR14097664  524288000  545259520  0 
none |   107110   000 none
gracedays  sharedhome USR   774858944  524288000  775946240  0   5 
days |  1115717   000 none
nametoolong sharedhome USR27418496  524288000  545259520  0 
none |11581   000 none

I was initially able to use

  df = pandas.read_csv(file_name, delimiter=r"\s+")

because all the values for 'grace' were 'none'.  Now, however,
non-"none" values have appeared and this fails.

I can't use

  pandas.read_fwf

even with an explicit colspec, because the names in the first column
which are too long for the column will displace the rest of the data to
the right.

The report which produces the file could in fact also generate a
properly delimited CSV file, but I have a lot of historical data in the
readable but poorly parsable format above that I need to deal with.

If I were doing something similar in the shell, I would just pipe the
file through sed or something to replace '5 days' with, say '5_days'.
How could I achieve a similar sort of preprocessing in Python, ideally
without having to generate a lot of temporary files?

Cheers,

Loris

--
This signature is currently under constuction.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hBypaGqqmBaUa_w_PNTK9VelYEJCChO6c7d8k1yz6N56806CJ0wtAfLhvj5UaWrGaccJTzKxrjQJCil9DJ470VZWO4fOfhk$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is mypy failing here

[Weatherby,Gerard]

https://github.com/python/mypy/issues/12971

From: Python-list  on 
behalf of Thomas Passin 
Date: Thursday, November 24, 2022 at 7:36 AM
To: python-list@python.org 
Subject: Re: is mypy failing here
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 11/24/2022 5:05 AM, Robin Becker wrote:
> I haven't used dataclasses or typing very much, but while playing about
> I found this didn't give me an expected error
>
> (.py312) robin@minikat:~/devel/reportlab
> $ cat tmp/examples/tdc.py && python tmp/examples/tdc.py && mypy
> tmp/examples/tdc.py
> ##
> from dataclasses import dataclass
>
> @dataclass
> class DC:
>  a: str
>  b: str
>
> def main():
>  dc = DC(DC, "B")
>  print(dc)
>
> if __name__ == "__main__":
>  main()
> ##
> DC(a=, b='B')
> Success: no issues found in 1 source file
> (.py312) robin@minikat:~/devel/reportlab
>
> DC.a is supposed to be a str and I expected mypy to indicate a type error
>
> should typing work for this case?
> --
> Robin Becker


Sounds like a bug.  With Python 3.10:

C:\temp\python>py -V
Python 3.10.4

C:\temp\python>py tdc.py
DC(a=, b='B')

C:\temp\python>mypy tdc.py
tdc.py:10: error: Argument 1 to "DC" has incompatible type "Type[DC]";
expected "str"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kFNV-Nh17mkih_KN3698Xv9rdmHpmQyEjpm-GBtPKVScpGjUISLM12MJ6yKj4O3JKeNVh7Ft9zhsPV8cURU$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: argparse — adding a --version flag in the face of positional args

[Weatherby,Gerard]

Use two parsers:

import argparse
import sys

vparser = argparse.ArgumentParser(add_help=False)
vparser.add_argument('--version',action="store_true",help="show version")
# look for version, ignore remaining arguments
vargs, _ = vparser.parse_known_args()
if vargs.version:
print("Version 2.0")
sys.exit(0)
parser = argparse.ArgumentParser()
parser.add_argument("positional",type=int)
# add version again, so it displays if --help called
parser.add_argument('--version',action="store_true",help="show version")
args = parser.parse_args()
# double argument
print(args.positional * 2)


From: Python-list  on 
behalf of Skip Montanaro 
Date: Sunday, November 27, 2022 at 6:42 PM
To: Python 
Subject: argparse — adding a --version flag in the face of positional args
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I have a script to which I'd like to add a --version flag. It should print
the version number then exit, much in the same way --help prints the help
text then exits. I haven't been able to figure that out. I always get a
complaint about the required positional argument.

I think I could use something like nargs='*', but that would push off
detection of the presence of the positional arg to the application.
Shouldn't I be able to tell argparse I'm going to process --verbose, then
exit?

Thx,

Skip
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!k-JSWNRKr8fNARGIFw3z_eh_Kv0ouXZKTDEQfWplA3Y3yrLUl81TmbNLiuDiXGOjgXcmNFPOqU2Ldmsh1VCLvLsxBas$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: argparse — adding a --version flag in the face of positional args

[Weatherby,Gerard]

More better:


import argparse

parser = argparse.ArgumentParser()
parser.add_argument("positional",type=int)
parser.add_argument('--version',action="version",version="2.0")
args = parser.parse_args()
# double argument
print(args.positional * 2)


From: Python-list  on 
behalf of Weatherby,Gerard 
Date: Sunday, November 27, 2022 at 10:29 PM
To: Skip Montanaro , Python 
Subject: Re: argparse — adding a --version flag in the face of positional args
Use two parsers:

import argparse
import sys

vparser = argparse.ArgumentParser(add_help=False)
vparser.add_argument('--version',action="store_true",help="show version")
# look for version, ignore remaining arguments
vargs, _ = vparser.parse_known_args()
if vargs.version:
print("Version 2.0")
sys.exit(0)
parser = argparse.ArgumentParser()
parser.add_argument("positional",type=int)
# add version again, so it displays if --help called
parser.add_argument('--version',action="store_true",help="show version")
args = parser.parse_args()
# double argument
print(args.positional * 2)


From: Python-list  on 
behalf of Skip Montanaro 
Date: Sunday, November 27, 2022 at 6:42 PM
To: Python 
Subject: argparse — adding a --version flag in the face of positional args
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I have a script to which I'd like to add a --version flag. It should print
the version number then exit, much in the same way --help prints the help
text then exits. I haven't been able to figure that out. I always get a
complaint about the required positional argument.

I think I could use something like nargs='*', but that would push off
detection of the presence of the positional arg to the application.
Shouldn't I be able to tell argparse I'm going to process --verbose, then
exit?

Thx,

Skip
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!k-JSWNRKr8fNARGIFw3z_eh_Kv0ouXZKTDEQfWplA3Y3yrLUl81TmbNLiuDiXGOjgXcmNFPOqU2Ldmsh1VCLvLsxBas$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!k-JSWNRKr8fNARGIFw3z_eh_Kv0ouXZKTDEQfWplA3Y3yrLUl81TmbNLiuDiXGOjgXcmNFPOqU2Ldmsh1VCLvLsxBas$><https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!k-JSWNRKr8fNARGIFw3z_eh_Kv0ouXZKTDEQfWplA3Y3yrLUl81TmbNLiuDiXGOjgXcmNFPOqU2Ldmsh1VCLvLsxBas$%3chttps:/urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!k-JSWNRKr8fNARGIFw3z_eh_Kv0ouXZKTDEQfWplA3Y3yrLUl81TmbNLiuDiXGOjgXcmNFPOqU2Ldmsh1VCLvLsxBas$%3e>
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iuJxNp5rr6BjU2VBDXr3OC1kal6NmqPTePUyYJ3K9gvrkpd-O6LrEW77sZ1Km5k3eglgSURIu991H8zLO9n2APmf$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iuJxNp5rr6BjU2VBDXr3OC1kal6NmqPTePUyYJ3K9gvrkpd-O6LrEW77sZ1Km5k3eglgSURIu991H8zLO9n2APmf$>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script not letting go of files

[Weatherby,Gerard]

Does the script exit when complete?

If you’re running on a Linux based system and have root access you can use lsof 
to see what processes have with files open. (Or use the psutil Python package).

From: Python-list  on 
behalf of Mike Dewhirst 
Date: Tuesday, November 29, 2022 at 2:20 AM
To: python-list@python.org 
Subject: Python script not letting go of files
I have a script which fetches a production site directly from a
Subversion repo using svn export

It runs a bunch of commands by calling this little method ...

def trycmd(cmd, log):
 retcode = -1
 ret = f"Trying {cmd}"
 try:
 retcode = os.system(cmd)
 ret = f"\n{cmd} -ok-> {retcode}"
 except Exception as err:
 ret = f"\n{cmd} -fail-> {err}"
 log.write(remove_password(ret))
 return retcode

This is the fetching script (omitting variables at the top) which
appears to be keeping a finger on files which Apache wants.

with open(fetchlog, 'a') as log:
 ret = f"\n\nFetching {tag}"
 log.write(ret)
 cmd = f"sudo rm -Rf {site_root}"
 if trycmd(cmd, log) == 0:
 cmd = f"sudo svn export --force --username {usr} --password
{pw} {svn_repo} {site_root}"
 if trycmd(cmd, log) == 0:
 # get any new dependencies
 cmd = f"sudo -H pip install -r {reqfile}"
 if trycmd(cmd, log) == 0:
 # run any migrations shipped from the repo
 cmd = f"sudo python3 {site_root}/manage.py migrate
--noinput --settings={settings}"
 if trycmd(cmd, log) == 0:
 # shouldn't find anything
 cmd = f"sudo python3 {site_root}/manage.py
makemigrations --noinput --settings={settings}"
 if trycmd(cmd, log) == 0:
 # should have been done already
 cmd = f"sudo python3 {site_root}/manage.py
migrate --noinput --settings={settings}"
 if trycmd(cmd, log) == 0:
 # remove all static files from their Apache dir
 cmd = f"sudo rm -Rf {static_root}/*"
 if trycmd(cmd, log) == 0:
 # copy all static files to the Apache
location
 cmd = f"sudo python3
{site_root}/manage.py collectstatic --noinput --settings={settings}"
 if trycmd(cmd, log) == 0:
 # set all permissions
 cmd = f"sudo {scripts}/perms_{host}.sh"
 if trycmd(cmd, log) == 0:
 cmd = "sudo service apache2
restart"
 if trycmd(cmd, log) == 0:
 ret = f"\nFinish {tag}\n\n"
 log.write(ret)
 else:
 print("Apache didn't restart")
 else:
 print("Didn't set permissions")
 else:
 print("Didn't collectstaic")
 else:
 print("Didn't delete static files")
 else:
 print("Didn't migrate 2")
 else:
 print("Didn't makemigration")
 else:
 print("Didn't migrate 1")
 else:
 print("Didn't install requirements")
 else:
 print("Didn't get source")
 else:
 print("Didn't remove site")
exit()

The problem I'm trying to fix is that after an Apache reload Apache
seems hellbent on filling up its scoreboard and running out of
resources. The kind folk on the Apache mailing list say something is
hogging the workers.

You can see the last operation above is an Apache restart. It should be
an Apache reload. Reload however lets Apache run out of resources.

Do any of you Python folks see any blunders in the above code along the
lines of not letting go of py files or static assets?

mod_wsgi is configured with 100% defaults.
mod_mpm_event conf per advice from the Apache mailing list is ...


 ServerLimit 32
 StartServers16
 MinSpareThreads 400
 MaxSpareThreads 800
 ThreadLimit 64
 ThreadsPerChild 50
 AsyncRequestWorkerFactor2
 MaxRequestWorkers   800
 MaxConnectionsPerChild  0


Server Version: Apache/2.4.52 (Ubuntu 2022.04) OpenSSL/3.0.2
mod_wsgi/4.9.0 Python/3.10
Server MPM: event
Server Built: 2022-09-30T04:09:50

DigitalOcean droplet 8BG RAM and lightly loaded.

Many thanks for an

Re: Python script not letting go of files

[Weatherby,Gerard]

"Weatherby,Gerard"  writes:
>Do any of you Python folks see any blunders in the above code along the
>lines of not letting go of py files or static assets?

Er, no, I just replied to the original poster.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Vb6 type to python

[Weatherby,Gerard]

Look at struct package: https://docs.python.org/3/library/struct.html


From: Python-list  on 
behalf of luca72.b...@gmail.com 
Date: Wednesday, November 30, 2022 at 11:48 AM
To: python-list@python.org 
Subject: Vb6 type to python
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hello i have a byte file, that fill a vb6 type like:
Type prog_real
codice As String * 12'hsg
denom  As String * 24'oo
codprof As String * 12   'ljio
note As String * 100
programmer As String * 11
Out As Integer
b_out As Byte'TRUE = Sec   FALSE= mm
asse_w As Byte   '3.zo Asse --> 0=Z  1=W
numpassi  As Integer 'put
len As Long  'leng
p(250) As passo_pg
vd(9) As Byte'vel.
qUscita(9) As Integer'quote
l_arco As Long   'reserved
AxDin As Byte'dime
End Type

How i can convert to python
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ky-bqK3l3Sbj0O_n3_x6Yo2wFaF5xABKKYgjbIPH49rdLZ2W_vQlW2gGbSQ7uRplRBZn_wS4h4evqcYQZaR1aSNJaTXnRQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Calling pselect/ppoll/epoll_pwait

[Weatherby,Gerard]

Signalfd and select could probably be made to work if one codes around the race 
conditions pselect is provided to avoid. I’m not sure if using the GIL (for 
CPython) is sufficient or if a dedicated concurrency control (e.g. lock, mutex, 
semaphore) is necessary.


An alternative is to call the C library function via a wrapper. I had need to 
use setfsuid on a project a couple of years ago and found this example:

https://gist.github.com/atdt/ebafa299e843a767139b



From: Python-list  on 
behalf of Barry 
Date: Friday, December 2, 2022 at 7:02 PM
To: Ian Pilcher 
Cc: python-list@python.org 
Subject: Re: Calling pselect/ppoll/epoll_pwait
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

> On 2 Dec 2022, at 20:03, Ian Pilcher  wrote:
>
> Does Python provide any way to call the "p" variants of the I/O
> multiplexing functions?
>
> Looking at the documentation of the select[1] and selectors[2] modules,
> it appears that they expose only the "non-p" variants.
>
> [1] 
> https://urldefense.com/v3/__https://docs.python.org/3/library/select.html__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_OD5qBjeA$
> [2] 
> https://urldefense.com/v3/__https://docs.python.org/3/library/selectors.html__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_MwNjgO8A$

Can you use signalfd and select/poll/epoll?

Barry

>
> --
> 
> Google  Where SkyNet meets Idiocracy
> 
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_PhasKBfg$
>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hClFFo4XdiwQAhHxDbHA5zFr490Of9uheHSf84V9cREMHyw1kX-baG5HzXWMt-hFLP30q6DpSUb2G_PhasKBfg$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: on the python paradox

[Weatherby,Gerard]

I use asyncio in a couple of places. Haven’t quite grokked it yet, though.

From: Python-list  on 
behalf of Stefan Ram 
Date: Wednesday, December 7, 2022 at 12:28 PM
To: python-list@python.org 
Subject: Re: on the python paradox
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

David Lowry-Duda  writes:
>I wonder what the appropriately esoteric language is today?

  How many Python programmers grog metaclasses or asyncio?
  So, I'd say: The two streams have converged. Python is the
  esoteric mainstream language.


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mvBtuPjNnWaXWGLt8aXlimysNpjtW4fP_ls5-vLPv8qWMyafvYcP6sgo0jcV7ngjmV1E3F0zJ5ipXlbjkBF8_l8$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?

[Weatherby,Gerard]

I’m not understanding the task. The sample code given is converting the input 
r’\x0a’ to a newline, it appears.


import re


def exam(z):
print(f"examine {type(z)} {z}")
for c in z:
print(f"{ord(c)} {c}")

s0 = r'\x0a'

def to1byte(matchobj):
return chr(int('0x' + matchobj.group(1), 16))
s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
exam(s0)
exam(s1)

---
examine  \x0a
92 \
120 x
48 0
97 a
examine 

10

From: Python-list  on 
behalf of Jach Feng 
Date: Wednesday, December 7, 2022 at 9:27 PM
To: python-list@python.org 
Subject: Re: How to convert a raw string r'xdd' to 'xdd' more gracefully?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Peter Otten 在 2022年12月8日 星期四清晨5:17:59 [UTC+8] 的信中寫道：
> On 07/12/2022 03:23, Jach Feng wrote:
> > s0 = r'\x0a'
> > At this moment it was done by
> >
> > def to1byte(matchobj):
> > return chr(int('0x' + matchobj.group(1), 16))
> > s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
> >
> > But, is it that difficult on doing this simple thing?
> >>> import codecs
> >>> codecs.decode(r"\x68\x65\x6c\x6c\x6f\x0a", "unicode-escape")
> 'hello\n'
Thank you. What I really want to handle is to any r'\xdd'. The r'\x0a' is for 
example. Sorry, didn't describe it clearly:-)
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kUnextA7_cF7EoP_4hGzC5Jq2wRvn8nwLwT8wmeNkgVjK_n6VG19fxb-4SwmDMwepWe8_bGaH9Y2LlkSvFRz$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?

[Weatherby,Gerard]

That’s actually more of a shell question than a Python question. How you pass 
certain control characters is going to depend on the shell, operating system, 
and possibly the keyboard you’re using. (e.g. https://www.alt-codes.net).

Here’s a sample program. The dashes are to help show the boundaries of the 
string

#!/usr/bin/env python3
import argparse
import logging


parser = 
argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('data')
args = parser.parse_args()
print(f'Input\n: -{args.data}- length {len(args.data)}')
for c in args.data:
print(f'{ord(c)} ',end='')
print()


Using bash on Linux:

./cl.py '^M
'
Input
  -
- length 3
13 32 10


From: Python-list  on 
behalf of Jach Feng 
Date: Thursday, December 8, 2022 at 9:31 PM
To: python-list@python.org 
Subject: Re: How to convert a raw string r'xdd' to 'xdd' more gracefully?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Jach Feng 在 2022年12月7日 星期三上午10:23:20 [UTC+8] 的信中寫道：
> s0 = r'\x0a'
> At this moment it was done by
>
> def to1byte(matchobj):
> return chr(int('0x' + matchobj.group(1), 16))
> s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)
>
> But, is it that difficult on doing this simple thing?
>
> --Jach
The whold story is,

I had a script which accepts an argparse's positional argument. I like this 
argument may have control character embedded in when required. So I make a post 
"How to enter escape character in a positional string argument from the command 
line? on DEC05. But there is no response. I assume that there is no way of 
doing it and I have to convert it later after I get the whole string from the 
command line.

I made this convertion using the chr(int(...)) method but not satisfied with. 
That why this post came out.

At this moment the conversion is done almost the same as Peter's 
codecs.decode() method but without the need of importing codecs module:-)

def to1byte(matchobj):
return matchobj.group(0).encode().decode("unicode-escape")
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hcg9ULzmtVUzMJ87Emlfsf6PGAfC-MEzUs3QQNVzWwK4aWDEtePG34hRX0ZFVvWcqZXRcM67JkkIg-l-K9vB$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: New computer, new Python

[Weatherby,Gerard]

Python in an IDE is much easier in the long run. We use PyCharm – there’s a 
free version: https://www.jetbrains.com/pycharm/download/#section=windows

From: Python-list  on 
behalf of DFS 
Date: Friday, December 9, 2022 at 4:36 PM
To: python-list@python.org 
Subject: Re: New computer, new Python
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 12/9/2022 12:13 PM, ker...@polaris.net wrote:
>
>
> Hello.  I've downloaded the new Python to my new Computer,  and the new 
> Python mystifies me.
>
> Instead of an editor, it looks like a Dos executable program.

python.exe is a Windows executable.



> How can I write my own Python Functions and subroutines in the new Python?

Open a text editor and write your own functions and subs.  Save the file
as prog.py.

 From the command line (not from inside the Python shell), type:

$ python prog.py



> It is version 3.11 (64 bit).

The latest and greatest.  Significantly sped up vs 3.10.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ncQETnsgLM_I29jKwV8WUZPhvn7g3Q5jBnkT_S0CmOf0KCORpl5xSLtMM8ZtE8gMsORNxCKhq01pCww$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Subtracting dates to get hours and minutes

[Weatherby,Gerard]

The difference between two datetime objects is a timedelta object. 
https://docs.python.org/3/library/datetime.html#timedelta-objects . It has a 
total_seconds() method.

This is a simple task, unless one of the datetimes has a time zone specified 
and the other doesn’t.

From: Python-list  on 
behalf of Marc Lucke 
Date: Monday, December 12, 2022 at 11:37 AM
To: python-list@python.org 
Subject: Re: Subtracting dates to get hours and minutes
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

my approach would be to convert your two date/times to seconds from
epoch - e.g.
https://urldefense.com/v3/__https://www.geeksforgeeks.org/convert-python-datetime-to-epoch/__;!!Cn_UX_p3!hBXQeMkZ3QYS6BI0yTHsADseWTXcDXOhKFkg35NnRMicvYQvwLo9c_ihSaTrG60LywsKQm6UNd7mAAYr$
  - then
subtract the number, divide the resultant by 3600 (hours) & get the
modulus for minutes.  There's probably a standard function - it should
be /very/ easy to do.

- Marc

On 12/12/2022 5:01 pm, Steve GS wrote:
> How do I subtract two time/dates and calculate the hours and minutes
> between?
> Steve
>
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hBXQeMkZ3QYS6BI0yTHsADseWTXcDXOhKFkg35NnRMicvYQvwLo9c_ihSaTrG60LywsKQm6UNSbh5q0S$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Keeping a list of records with named fields that can be updated

[Weatherby,Gerard]

I have a lot of NamedTuples in my codebase, and I now add new ones never. They 
were a good option prior to Python 3.7 but dataclasses are much easier to work 
with and are almost a drop-in substitute.

A combination of a default dictionary and a dataclass might meet your needs:



import collections
from dataclasses import dataclass


@dataclass
class AccountingEntry:
description: str
# other fields


ledger = collections.defaultdict(list)

for ae in get_accounting_entries():
ledger[ae.description] = ae



From: Python-list  on 
behalf of songbird 
Date: Wednesday, December 14, 2022 at 10:38 PM
To: python-list@python.org 
Subject: Keeping a list of records with named fields that can be updated
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

  I'm relatively new to python but not new to programming in general.

  The program domain is accounting and keeping track of stock trades and other 
related information (dates, cash accounts, interest, dividends, transfers of 
funds, etc.)

  Assume that all data is CSV format.  There are multiple files.

  Assume there is a coherent starting point and that all data is in order.

  Assume each line contains a description.  The description determines what the 
line is.  The number of fields in the line does not change within the data file 
but it may happen that later lines in other files may be different other than 
the fact that they all must contain a description.

  All descriptions are deterministic (none are recursive or referencing things 
from the future).  All things referenced in the description which do not 
already exist are added to a list (or perhaps more than one in a few cases) and 
may contain some basic information (the date, how many and for how much, or a 
total amount or a fee or ...)  If the field of the line isn't a number it is 
either a symbol or a description.

  A default action is simply to keep most parts of the line and to adjust any 
totals of a previously seen description that matches by whatever amounts are on 
the line.  The key is the description.

  I've already written one program based upon the files I already have which 
works but what happens is that new descriptions are added (new accounts, new 
stocks, etc.) and I don't want to have to write new code manually every time a 
description changes.

  I started using named tuples (it works for reading in the files and accessing 
the fields) but I cannot update those so I need to use something else to give 
me the list of unique descriptions and fields that I need to update.  I've not 
gotten beyond that yet as I'm still learning.

  Suggestions?

  Thanks!  :)


  songbird
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ndkoYjlClLoELvhzTpXFZEtJ70fXjdFllo-ce0fJ4f0AdRLQXvryO11ZSJ16tf-Ke-pko3kmBxW1cesvrQAQUQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Single line if statement with a continue

[Weatherby,Gerard]

I once saw a C function full of GOTOs which jumped to the return state at the 
bottom of the functions because the programmer had learned that “multiple 
returns are a bad idea.”

I totally agree multiple returns causing confusion is a symptom of poor design, 
not a cause.

Required cleanup is easily handled by the try / finally construct.

From: Python-list  on 
behalf of avi.e.gr...@gmail.com 
Date: Thursday, December 15, 2022 at 2:07 PM
To: python-list@python.org 
Subject: RE: Single line if statement with a continue
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Multiple returns is not always a problem as it depends on the nature of a
task whether it has complex enough cases.

I have seen code that instead sets Boolean variables when it is ready to
return and everything else keeps checking the variables to skip further
processing so the program then slides down to a single return statement. If
done properly, it boils down to the same result if VERY carefully done but
with lots of sometimes complex IF statements that may be not be updated well
if the logic changes a bit.

In such cases, I vastly prefer clean and unambiguous returns  from the
function right at the point where the decision is made, UNLESS the exit is
not always clean as there may be cleanup or finalization of some kind
required that is best done at a single point.

If efficiency is an issue, then clearly a rapid exit may beat one where
processing continues for a while and also often beats a method that involves
creating and calling multiple smaller functions with lots of overhead.

Having said all that, of course, if you can find a fairly simple algorithm
that only returns from one place, use it instead of a convoluted one. The
issue is not necessarily that multiple return points are bad, but that they
are often a symptom of sloppy planning. But for some problems, they fit well
and simplify things.

-Original Message-
From: Python-list  On
Behalf Of Stefan Ram
Sent: Thursday, December 15, 2022 7:42 AM
To: python-list@python.org
Subject: Re: Single line if statement with a continue

Chris Green  writes:
>I always try to avoid multiple returns from functions/methods, as soon
>as things get complex it's all to easy to miss clean-up etc.

  This "complexity" could also mean that the function has
  become too large. In such a case, one could say that the
  /size/ of the function is the actual cause of problems
  and not multiple returns.

|Fools ignore complexity. Pragmatists suffer it. Some can avoid it.
|Geniuses remove it.
Alan Perlis (1922/1990)

  Within a small function, multiple returns are rarely
  a problem.

  When a function is large, one can apply well-known
  refactors. For an example, look at the code in the
  first post of the thread

Python script not letting go of files
Date: Tue, 29 Nov 2022 12:52:15 +

  and then at my reply of

29 Nov 2022 14:44:39 GMT.

>"No multiple returns" is often found in programming guidelines.

  I religiously followed that when I did more C programming
  than today. Then, I read an article about how the result
  pattern makes functions measurably slower. (It should not
  with an optimizing compiler, but it did due to those
  measurements. Can't find that article now, though.)

  "Result pattern" I call writing,

if a:
result = 123
else:
result = 456
return result

  , instead of,

if a:
return 123
else
return 456

  .


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!lVeLOl91qPUjowC1ch_u353upn8X-V4rsReaNberWpIXBlBP6CYcDgr_aaMb0ZHoYX4YWO8id1biCn6sW7V6vJM$

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!lVeLOl91qPUjowC1ch_u353upn8X-V4rsReaNberWpIXBlBP6CYcDgr_aaMb0ZHoYX4YWO8id1biCn6sW7V6vJM$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Subtracting dates to get hours and minutes

[Weatherby,Gerard]

Not sure what you mean:

x = datetime.datetime(year=2018,month=12,day=4,hour=10)
y = datetime.datetime(year=2022,month=12,day=13,hour=5)
print(x -y)


-1470 days, 5:00:00

If you want to display years, (x-y).days /365

From: Python-list  on 
behalf of Gronicus@SGA.Ninja 
Date: Thursday, December 15, 2022 at 5:02 PM
To: 'anthony.flury' , python-list@python.org 

Subject: RE: Subtracting dates to get hours and minutes
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Yes, it works like a charm. On the tupility of it all.
Special thanks for the explanation too…..



Now that the code no longer produces the errors, I see that the year and month 
not included in the calculation? How do I fix this?



From: anthony.flury 
Sent: Thursday, December 15, 2022 1:47 PM
To: Gronicus@SGA.Ninja
Subject: RE: Subtracting dates to get hours and minutes



What is likely happening is that when you read the data from the file you are 
not reading a tuple, you are reading a 26 charcter string.

You have to convert that string into a tuple - the easiest way will be somthing 
like this :



timet = tuple(int(x.strip()) for x in timestring[1:-1].split(','))



where timestring is the data you get from the file

The [1:-1] removes the () from the data

The .split(",") method creates a temporary list from the remaining string 
breaking the string where there are commas

The x.strip() removes spaces from each item in the temporary list.



Note that the * unpack operator doesn't just unpack tuples, it works on an 
iterable, so when you read the data from the file currently, and then use * on 
it, it will pass 26 separate characters to the function.





-- Original Message --
From: Gronicus@SGA.Ninja 
To: "'Thomas Passin'" mailto:li...@tompassin.net> >; 
python-list@python.org 
Sent: Thursday, 15 Dec, 22 At 18:14
Subject: RE: Subtracting dates to get hours and minutes

So far so good , I can now use a variable in datetime.datetime but it only
works if I hard-code the time/date information. Now I want to have the code
read from a file but I get: TypeError: function takes at most 9 arguments
(26 given)

I figure that the structure in the file is incorrect. What should it be? The
entry in the file is (2022, 12, 13, 5, 3, 30) but when my program tries to
use it I get the error.

The program is a bit more sophisticated now but here is the update with a
sample of the SPECIFICATIONS.txt file:
=

# This program compares two Timedate values, subtracts the two and
# converts the difference to seconds and hours.
#

# %A Monday # %a Mon # %B January # %b Jan
# %d 05 day # %m month as 01 # %Y 2020 # %y 20
# %H 24 # %I 12 # %M 30 min # %S Seconds

import time
import datetime
from time import gmtime, strftime ##define strftime as time/date right now
# ==

def GetSpecByItem(GetThisOne): #get line by item in column 4 - 7
ItemValue = "--"

with open("SPECIFICATIONS.txt" , 'r') as infile:
for lineEQN in infile: # loop to find each line in the file for that
dose
if ((lineEQN[4:7]== GetThisOne)):
ItemValue = lineEQN[30:60].strip() # Just the Data
return(ItemValue)

"""
SPECIFICATIONS.txt

IYf HRB Humalog R Date (2018, 12, 4, 10, 7, 00) ##
IYf HRG Humulin R Date (2022, 12, 13, 5, 3, 30) ##
"""
# == Main() ==
print()
Startt = "404"
Stopp = "404"

Answer = "Y"
Answer = input("Run test A? (" + Answer + ")" )

if Answer == "Y" or Answer == "y" or Answer == "":
print()
print(" Running Test A:")
# Year Mth Day Hour Min Sec
Startt = 2018, 12, 4, 10, 7, 00
Stopp = 2022, 12, 12, 1, 15, 30
NowTime = 2022, 12, 14, 21, 15, 30
else:
print(" Running Test B:")
Startt = GetSpecByItem("HRG")
Stopp = GetSpecByItem("HRB")
NowTime = strftime("(%Y, %m, %d, %H, %M, %S)")

print()
print("55 NowTime = " + str(NowTime))
print("56 Startt = " + str(Startt))
print("57 Stopp = " + str(Stopp))
print()

NowTime = datetime.datetime(*NowTime)
Startt = datetime.datetime(*Startt)
Stopp = datetime.datetime(*Stopp)

#Start == Startt # True"
#print("Startt test = " + Start)
# =
print()
c = NowTime - Stopp
minutes = c.total_seconds() / 60
minutes = c.seconds / 60
hours = 0

while (minutes > 59):
minutes = minutes - 60
hours += 1
minutes = round(minutes)
print ("77 Hours = <" + str(hours) + ">")
print ("78 Minutes = <" + str(minutes) + ">")
if hours > 7:
print(" Time to inject Humulin R u500.")

pause = input("Pause")
# ==


-Original Message-
From: Python-list mailto:python-list-bounces+gronicus=sga.ni...@python.org> > On
Behalf Of Thomas Passin
Sent: Tuesday, December 13, 2022 11:20 PM
To: python-list@python.org 
Subject: Re: Subtracting dates to get h

Re: String to Float, without introducing errors

[Weatherby,Gerard]

https://docs.python.org/3/library/decimal.html

Get Outlook for iOS

From: Python-list  on 
behalf of Paul St George 
Sent: Saturday, December 17, 2022 6:51:17 AM
To: python-list@python.org 
Subject: String to Float, without introducing errors

*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I have a large/long array of numbers in an external file. The numbers look like 
this:

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66
...

When I bring the numbers into my code, they are Strings. To use the numbers in 
my code, I want to change the Strings to Float type because the code will not 
work with Strings but I do not want to change the numbers in any other way.

So, I want my Strings (above) to be these numbers.

-64550.727
-64511.489
-64393.637
-64196.763
-63920.2
-63563.037
-63124.156
-62602.254
-61995.895
-61303.548
-60523.651
-59654.66
...

Please help!










--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!guE_zPsjxMW4k6nHIdOqZbrt8SdjUC9GELXgSHatARIr2PrAYr6tXCmixkZGNocjsf9SKLduQFjZjM7tOeaQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip/setuptools: Entry points not visible from pkexec-root-environment

[Weatherby,Gerard]

"sudo python3 -m pip
install -e ."

You’ve already started down a problematic road. I recommend installing root 
level Python packages through your system package manager. (apt for debian, or 
whatever RedHat is using now).

If a package you need isn’t available from the system level virtual 
environments are your friend.

I’ve never used pkexec. Generally, just use sudo. There’s an -E flag to 
preserve the environment. We generally write bash wrappers that set whatever 
environment we need.


From: Python-list  on 
behalf of c.bu...@posteo.jp 
Date: Sunday, December 18, 2022 at 12:56 PM
To: Python-list@python.org 
Subject: pip/setuptools: Entry points not visible from pkexec-root-environment
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hello,

when I install a package on a GNU/Linux system via "sudo python3 -m pip
install -e ."
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Installation hell

[Weatherby,Gerard]

Personally, I don’t use Windows and avoid it like the plague.  Python is easy 
to install on Linux and Mac.

I’d start here: 
https://learn.microsoft.com/en-us/visualstudio/python/overview-of-python-tools-for-visual-studio?view=vs-2022

From: Python-list  on 
behalf of Jim Lewis 
Date: Sunday, December 18, 2022 at 12:56 PM
To: Python-list@python.org 
Subject: Fwd: Installation hell
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I'm an occasional user of Python and have a degree in computer science.
Almost every freaking time I use Python, I go through PSH (Python Setup
Hell). Sometimes a wrong version is installed. Sometimes it's a path issue.
Or exe naming confusion: python, python3, phthon311, etc. Or library
compatibility issues - took an hour to find out that pygame does not work
with the current version of python. Then the kludgy PIP app and using a DOS
box under Windows with command prompts which is ridiculous. God only knows
how many novice users of the language (or even intermediate users) were
lost in the setup process. Why not clean the infrastructure up and make a
modern environment or IDE or something better than it is now. Or at least
good error messages that explain exactly what to do. Even getting this
email to the list took numerous steps.

-- A frustrated user
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gdOs-oC1JZcmvMXy1G4irRpzHCUmF565UXVdCjzSWNGZKpmZ04I_llDX4WUeob3asBCjLe6TIthAAhmwFgbph9u1m9A$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Installation hell

[Weatherby,Gerard]

I was suggesting a possible resource while making it clear I am not Windows 
savvy. My avoidance of Windows has nothing to do with Python, so there is no 
need to discuss it here.

If I don’t care which Python I’m using, I’ll type python3. If I want3.9,  
python3.9. python3 -V tells me the default on that system. (It varies across 
our servers).

If a Python package isn’t available in the mainline Ubuntu repository 
(python3-whatever), I’m creating a virtual environment. It just saves trouble 
in the long run. I have this in ~/bin to create an environment. I’m slowly 
moving projects to the pyproject.toml because pip is complaining about the 
setup.py installs.


#!/bin/bash
VERS=3.8 #default version
VENV=venv #default environment name
if [ $# -gt 0 ]; then
VERS=$1
fi
if [ $# -gt 1 ]; then
VENV=$2
fi
PYTHON=python$VERS
echo $PYTHON
$PYTHON -m venv $VENV
./$VENV/bin/pip install -U pip ipython
if [ -e requirements.txt ]; then
./$VENV/bin/pip install -r requirements.txt
fi
if [ -e setup.py ]; then
./$VENV/bin/python setup.py install
fi
if [ -e pyproject.toml ]; then
./$VENV/bin/pip install -e .
fi

From: Python-list  on 
behalf of Thomas Passin 
Date: Monday, December 19, 2022 at 11:05 AM
To: python-list@python.org 
Subject: Re: Installation hell
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 12/19/2022 9:59 AM, Weatherby,Gerard wrote:
> Personally, I don’t use Windows and avoid it like the plague.  Python is easy 
> to install on Linux and Mac.

That's not been my experience.  Windows installers for Python have
worked well for me over many generations of Python releases.  It's Linux
where I've found difficulties.  For example, if your distro's Python
install didn't include tkinter (or even pip), how do you get it?  It's
different for different Linux distros.  I generally have to use internet
searches to find out.

For another example, when you use pip to install a package, it sometimes
suggests that you install a newer version of pip itself. Should you do
that?  On Linux, probably not, because the distro will have modified pip
so it puts things in distro-specific places. Yet there is no newer
version of pip available through the distro's package manager.  Will
anything bad happen if you don't update pip?  Who knows?

I have a Linux VM that has several versions of Python3 on it.  Python3.8
came installed with the distro, but for some programs I need Python
3.9+.  If I forget which versions I have, how can I find out?  People
say to use which, but that doesn't work - it only reports "python3".
This does work, but it's not all that easy to remember (the grep "site"
part is just to filter out uninformative result lines):

~$ find 2>/dev/null ~ -name python -type d |grep "site"
/home/tom/.local/lib/python3.9/site-packages/PyQt5/Qt5/qsci/api/python
/home/tom/.local/lib/python3.8/site-packages/pandas/_libs/src/ujson/python
/home/tom/.local/lib/python3.10/site-packages/PyQt5/Qt5/qsci/api/python

Not that this task is much easier to remember on Windows, but it's not
harder.  One way: the "py" launcher will tell you:

py --list
-V:3.10 *Python 3.10 (64-bit)
-V:3.9   Python 3.9 (64-bit)
-V:3.7   Python 3.7 (64-bit)
-V:2.7

This is not Linux-bashing, but there's no need for Windows-bashing either.

> I’d start here: 
> https://learn.microsoft.com/en-us/visualstudio/python/overview-of-python-tools-for-visual-studio?view=vs-2022
>
> From: Python-list  on 
> behalf of Jim Lewis 
> Date: Sunday, December 18, 2022 at 12:56 PM
> To: Python-list@python.org 
> Subject: Fwd: Installation hell
> *** Attention: This is an external email. Use caution responding, opening 
> attachments or clicking on links. ***
>
> I'm an occasional user of Python and have a degree in computer science.
> Almost every freaking time I use Python, I go through PSH (Python Setup
> Hell). Sometimes a wrong version is installed. Sometimes it's a path issue.
> Or exe naming confusion: python, python3, phthon311, etc. Or library
> compatibility issues - took an hour to find out that pygame does not work
> with the current version of python. Then the kludgy PIP app and using a DOS
> box under Windows with command prompts which is ridiculous. God only knows
> how many novice users of the language (or even intermediate users) were
> lost in the setup process. Why not clean the infrastructure up and make a
> modern environment or IDE or something better than it is now. Or at least
> good error messages that explain exactly what to do. Even getting this
> email to the list took numerous steps.
>
> -- A frustrated user
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo

Re: pygame.midi input/output not working

[Weatherby,Gerard]

https://github.com/pygame/pygame/issues/3522

From: Python-list  on 
behalf of Peter J. Holzer 
Date: Thursday, December 22, 2022 at 5:06 AM
To: python-list@python.org 
Subject: Re: pygame.midi input/output not working
On 2022-12-21 17:23:47 -0500, Thomas Passin wrote:
> The pygame web site says this:
>
> "Pygame still does not run on Python 3.11"

This doesn't sound like "we haven't got around to preparing packages
yet" and more like "there's a serious incompatibility we haven't solved
yet".

Does anybody know what the issue is?

hp

--
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: NoneType List

[Weatherby,Gerard]

Just use the addition operator:

a = [1,2]

a = a + [3,4]

a is now [1, 2, 3, 4]

From: Python-list  on 
behalf of Goran Ikac 
Date: Saturday, December 31, 2022 at 1:53 AM
To: python-list@python.org 
Subject: NoneType List
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Happy New Year, everybody!
I'm new in the Python List, new in Python world, and new in coding.
A few days (weeks?) ago, I faced a problem trying to write a program for an
exercise. I asked for help and nobody answered.
In the meantime, I found a part of the solution, but a part still remains a
mystery for me. Please run this small snippet, and help.
Thanks

a = [1, 2]
print()

print("a ==", a)
print("type(a) is", type(a))

b = a.append(3)
print("\nb =", "a.append(3)")
print("b ==", b)
print("type(b) is", type(b))

c = ['1', '2']
print("\nc ==", c)
print("type(c) is", type(c))

d = c.append('3')
print("\nd =", "c.append('3')")
print("d ==", d)
print("type(d) is", type(d))

"""
I mean: why b = a.append(something) is the None type, and how to make a new
list
that contains all the items from a and some new items?
I wasn't able to solve it in a few hours :(
Now I know:
"""

crta = '='
print('\n', 4 * ' ', crta * (len('The solution:')), sep='')
print(3 * ' ', 'The solution:')
print(4 * ' ', crta * (len('The solution:')), sep='')
print('\nThe solution is the slice, an element of Python syntax that
allows')
print('to make a brand new copy of a list, or parts of a list.')
print('The slice actually copies the list\'s contents, not the list\'s
name:')

print()
a = [1, 2]
print("a ==", a)
print("type(a) is", type(a))

b = a[:]
print("\nb = a[:]")
print("b ==", b)
b.append(3)
print("\nb =", "b.append(3)")
print("b ==", b)
print("type(b) is", type(b))
print("\na ==", a)

print('But I still don't know why "b = a.append(something)" is the None
type.')
print('Is there anybody out there?!')
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mv_VSTDWgnB7T16IHcBh7TaT3whji-SsPeUtaBmHdSsIJknJZ16qeALOSjCkjCmqheE1pJ-47P_mwAauV-0TjQo$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Weatherby,Gerard]

Really doesn’t have much to do with Python and very much with standard streams, 
which go back as far as the 1950s. 
https://en.wikipedia.org/wiki/Standard_streams

When a user types -h / --help to a Python argparse script, the output of the 
script is the help message. The standard behavior is to print that and exit, 
regardless of any other arguments that are passed. So its behavior is 
consistent with the concept of standard out.

From: Python-list  on 
behalf of MRAB 
Date: Tuesday, January 3, 2023 at 11:41 AM
To: python-list@python.org 
Subject: Re: What should go to stdout/stderr and why Python logging write 
everything to stderr?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2023-01-03 15:35, c.bu...@posteo.jp wrote:
> Hello,
>
> this posting isn't about asking for a technical solution. My intention
> is to understand the design decision Python's core developers made in
> context of that topic.
>
> The logging module write everything to stderr no matter which logging
> level is used.
>
> The argparse module does it more differentiated. If arguments are
> mandatory but none are given then argparse produce a short version of
> the usage info; on stderr. If the user request the usage info via "-h"
> it goes to stdout. This is the behavior I would expect.
>
> Why does logging behave different? DEBUG and INFO imho should go to
> stdout not stderr.
>
> Of course I could modify the handlers etc. to workaround this. But I
> assume that there was something in mind of the Python developers when
> they decided that.
>
> My goal is not to divide between the use of print() or logging.info()
> in my own code. This would mess up a lot.

The purpose of stderr is to display status messages, logging and error
messages, even user prompts, and not mess up the program's actual
output. This is important on a *nix system where you might be piping the
output of one program into the input of another.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jni-tegcOXxLEkj6ohH0wbjz0Gd2OmHirJln0S1hX5z3nT6MUnqw4-ZMgaJGkzT228O9zSczGws4tyRERCMntNGR$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Weatherby,Gerard]

If sys.stdout is a tty, it typically flushes on newline. e. g.

!/usr/bin/env python3
import time
import sys
print("No flush",end='',file=sys.stdout)
time.sleep(2)
print("flushed",file=sys.stdout)
time.sleep(5)

will print the “flushed” 5 seconds before the script ends

From: Python-list  on 
behalf of Eryk Sun 
Date: Tuesday, January 3, 2023 at 1:33 PM
To: c.bu...@posteo.jp 
Cc: python-list@python.org 
Subject: Re: What should go to stdout/stderr and why Python logging write 
everything to stderr?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 1/3/23, c.bu...@posteo.jp  wrote:
>
> If the user request the usage info via "-h" it goes to stdout.

The standard file for application output is sys.stdout. Note that
sys.stdout may be buffered, particularly if it isn't a tty. When a
file is buffered, writes are aggregated and only written to the OS
file when the buffer fills up or is manually flushed.

> Why does logging behave different? DEBUG and INFO imho should go to
> stdout not stderr.

The standard file for error messages and other diagnostic information
is sys.stderr. This file should never be buffered.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jGBVNrRuUThZCKMmShbuvBgggwv7FBDL9h2vW-vvehPnBHdfkrJUhohhZCgsCAqlRrDluk9c526jABrLjg$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Weatherby,Gerard]

Dealing with stdout / stderr is bash is just syntax. I don’t always remember it 
off the top of my head but … stackoverflow.com.

On Linux at least it’s possible to pipe to arbitrary streams, it seems. The 
following uses bash to write “Hi” to the file “third” opened by Python. I 
determined the file was 5 empirically.


import os
import subprocess
command= 'echo Hi >/dev/fd/5'
fd = os.open("third",os.O_WRONLY|os.O_CREAT)
os.set_inheritable(fd,True)

subprocess.run(('/usr/bin/bash','-c',command),close_fds=False)

From: Python-list  on 
behalf of Michael Torrie 
Date: Tuesday, January 3, 2023 at 5:18 PM
To: python-list@python.org 
Subject: Re: What should go to stdout/stderr and why Python logging write 
everything to stderr?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***


Maybe some day an interface and shell syntax will be developed to
interact with an arbitrary number of standard streams.  Current piping
syntax really only works well with one stream and even trying to use
stderr and stdout with pipes and filters in a shell is awkward.




--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!j8qV9yx1DJI_G2F-q1fQz2LfnVYoMi40Qpk_h8bxrOcw50rVXpwScpFJSyZ212Tm9rj6T7vKgJjaIEgLRw$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: No solution for "--verbose" (on stdout) output in Pythonds standard library?

[Weatherby,Gerard]

A couple options. The -vvv is more of Linux thing rather than Pythonic, to my 
way of thinking.



import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-v',action='store_true')
parser.add_argument('-vv',action='store_true')
parser.add_argument('-vvv',action='store_true')
args = parser.parse_args()

parser = argparse.ArgumentParser()
parser.add_argument('-v','--verbose',type=int,default=0,nargs='?',help="verbose 
level")
args = parser.parse_args()
level = args.verbose if args.verbose is not None else 0
print(level)


Personally I just do:

import argparse
import logging


logging.basicConfig()
parser = argparse.ArgumentParser()
parser.add_argument('-l', '--loglevel', default='WARN', help="Python logging 
level")
args = parser.parse_args()
logger = logging.getLogger(__name__)
logger.setLevel(getattr(logging,args.loglevel))

From: Python-list  on 
behalf of c.bu...@posteo.jp 
Date: Wednesday, January 4, 2023 at 8:55 AM
To: python-list@python.org 
Subject: No solution for "--verbose" (on stdout) output in Pythonds standard 
library?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hello,

first I have to say that in my current and fresh humble opinion the
often seen "--verbose" switch in command line applications should
affect only the messages given to the users. This means messages on
"stdout". That is what this question is about.

The logging module is not an option because it works on stderr and is
not intended to offer messages for the user but just for the system and
its admins via logs (log-files, syslog, stderr redirection, ...).

Using logging handlers redirecting to stdout are considered as
workarounds by me and also not an option.

This is not only my opinion but also part of the Python documentation:
https://urldefense.com/v3/__https://docs.python.org/3/howto/logging.html*when-to-use-logging__;Iw!!Cn_UX_p3!gctSoo2C-FjIeI4wfVetRVylYZ-X1te71-Q05ylEpJ_2XICGGoFbXFjrm02smi-UKx0H2EbiEXiJLfNcgTsq$

I'm so detailed here about that difference between stdout and stderr
because in the wild (e.g. on StackOverflow) you often find "use logging
log levels" as a solution for that problem, which IMHO isn't one.

Now the question:
>From my research on the docs it seems there is no feature in the
standard library which could help me to implement "--verbose" or
multiple verbosity levels like "-vvv"?
I found some workarounds/hacks.
https://urldefense.com/v3/__https://stackoverflow.com/q/5980042/4865723__;!!Cn_UX_p3!gctSoo2C-FjIeI4wfVetRVylYZ-X1te71-Q05ylEpJ_2XICGGoFbXFjrm02smi-UKx0H2EbiEXiJLU6LbUZG$
But my experience with Python as a Swiss knife is that there is always
a standard solution for such basic and often reinvented things. I won't
believe that each Python developer writes its own verbose feature. ;)
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gctSoo2C-FjIeI4wfVetRVylYZ-X1te71-Q05ylEpJ_2XICGGoFbXFjrm02smi-UKx0H2EbiEXiJLYWBuS4g$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should go to stdout/stderr and why Python logging write everything to stderr?

[Weatherby,Gerard]

logging.basicConfig()
logging.info(“Nice to know”)
logging.debug(“Details for when things are funky”)
logging.warn(“Trouble is brewing”)

From: Python-list  on 
behalf of Grant Edwards 
Date: Thursday, January 5, 2023 at 3:31 PM
To: python-list@python.org 
Subject: Re: What should go to stdout/stderr and why Python logging write 
everything to stderr?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2023-01-05, Thomas Passin  wrote:

> The logging system is so configurable that...

I find it almost impossible to use unless I copy a working example I
find somewhere. ;)

I'm not at all surprised that the OP didn't understand how it
works. I've been writing Python programs for over 20 years, and it's
beyond me.

--
Grant




--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!l6vSXQFKppEuLS0R5gYeLYaiHyVFfs2Rapqm1oPGEtvnZ5ivQyApZcnJyNTnnH9zEVY80ajNb-HfYkNwgw8fMtsnlSOT$
-- 
https://mail.python.org/mailman/listinfo/python-list

Python logging (was Re: What should go to stdout/stderr and why Python logging write everything to stderr?)

[Weatherby,Gerard]

FWIW, it wasn’t too difficult to build a logging handler to send messages to 
Slack.

https://pypi.org/project/slack-webclient-logger/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: To clarify how Python handles two equal objects

[Weatherby,Gerard]

For clarification, equality is not identity in Python. e.g.

x = 7
y = 7.0
print(x == y)
print(x is y)

Will return
True
False

Full explanation at 
https://docs.python.org/3/reference/expressions.html#comparisons


From: Python-list  on 
behalf of Chris Angelico 
Date: Tuesday, January 10, 2023 at 3:47 PM
To: Python List 
Subject: Re: To clarify how Python handles two equal objects
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On Wed, 11 Jan 2023 at 07:41, Jen Kris  wrote:
>
>
> Thanks for your comments.  I'd like to make one small point.  You say:
>
> "Assignment in Python is a matter of object references. It's not
> "conform them as long as they remain equal". You'll have to think in
> terms of object references the entire way."
>
> But where they have been set to the same object, an operation on one will 
> affect the other as long as they are equal (in Python).  So I will have to 
> conform them in those cases because Python will reflect any math operation in 
> both the array and the matrix.
>

It's not that "an operation on one will affect the other" - it's that,
no matter how you refer to that object, you're affecting *that one
single object*. It's like when you're washing a window; the inside and
outside of the window are the exact same window, so regardless of
where you're looking at it from, it's the same single window and
changes affect it equally.

So you shouldn't have to replicate any changes. What should be
happening is that you find the right object to mutate, and mutate
that. For example:

stuff = [[1, 2, 3], [4, 5, 6]]
stuff.append(stuff[0])
print(stuff)

You now have two references to the same list, inside another list. Any
change to stuff[0] is a change to stuff[2], because they're the exact
same list. When you append "a reference to this list over here" (which
you found by asking for stuff[0]) to the outer list, you get that
list.

That's Python's object model, and trying to cheat it by copying
changes is just going to cause untold nightmares of desynchronization.

ChrisA
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iORRBJFui8Kn7WzY0ZRPfSdGKcmnDV81UffITsv7ExEAbBXEtv86qC3BOvGaDXCAY708Q4QbDXh0_wo7$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fast lookup of bulky "table"

[Weatherby,Gerard]

That’s about what I got using a Python dictionary on random data on a high 
memory machine.

https://github.com/Gerardwx/database_testing.git

It’s not obvious to me how to get it much faster than that.

From: Python-list  on 
behalf of Dino 
Date: Sunday, January 15, 2023 at 1:29 PM
To: python-list@python.org 
Subject: Re: Fast lookup of bulky "table"
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Thank you for your answer, Lars. Just a clarification: I am already
doing a rough measuring of my queries.

A fresh query without any caching: < 4s.

Cached full query: < 5 micro-s (i.e. 6 orders of magnitude faster)

Desired speed for my POC: 10  Hey,
>
> before you start optimizing. I would suggest, that you measure response
> times and query times, data search times and so on. In order to save
> time, you have to know where you "loose" time.
>
> Does your service really have to load the whole table at once? Yes that
> might lead to quicker response times on requests, but databases are
> often very good with caching themselves, so that the first request might
> be slower than following requests, with similar parameters. Do you use a
> database, or are you reading from a file? Are you maybe looping through
> your whole dataset on every request? Instead of asking for the specific
> data?
>
> Before you start introducing a cache and its added complexity, do you
> really need that cache?
>
> You are talking about saving microseconds, that sounds a bit as if you
> might be “overdoing” it. How many requests will you have in the future?
> At least in which magnitude and how quick do they have to be? You write
> about 1-4 seconds on your laptop. But that does not really tell you that
> much, because most probably the service will run on a server. I am not
> saying that you should get a server or a cloud-instance to test against,
> but to talk with your architect about that.
>
> I totally understand your impulse to appear as good as can be, but you
> have to know where you really need to debug and optimize. It will not be
> advantageous for you, if you start to optimize for optimizing's sake.
> Additionally if you service is a PoC, optimizing now might be not the
> first thing you have to worry about, but about that you made everything
> as simple and readable as possible and that you do not spend too much
> time for just showing how it could work.
>
> But of course, I do not know the tasks given to you and the expectations
> you have to fulfil. All I am trying to say is to reconsider where you
> really could improve and how far you have to improve.
>
>
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!npizb3UAz-jPUnhlimB3_lctLibK5EW4zJwjZVmQ41yV_-2WSm2eQ5cTi8vzOEuCfsdNTjIvIhFcakrX$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fast lookup of bulky "table"

[Weatherby,Gerard]

I think any peformance improvements would have to come from a language change 
or better indexing of the data.

From: Python-list  on 
behalf of Weatherby,Gerard 
Date: Sunday, January 15, 2023 at 2:25 PM
To: Dino , python-list@python.org 
Subject: Re: Fast lookup of bulky "table"
That’s about what I got using a Python dictionary on random data on a high 
memory machine.

https://urldefense.com/v3/__https://github.com/Gerardwx/database_testing.git__;!!Cn_UX_p3!keHKWsb1LGR6u_6BQA04MyEJlnzICq04FNdn8z9BnnjG8NopVu3KiL0k3rMiowxtp87xBUi6OcavBQIqksBjbd9v$<https://urldefense.com/v3/__https:/github.com/Gerardwx/database_testing.git__;!!Cn_UX_p3!keHKWsb1LGR6u_6BQA04MyEJlnzICq04FNdn8z9BnnjG8NopVu3KiL0k3rMiowxtp87xBUi6OcavBQIqksBjbd9v$>

It’s not obvious to me how to get it much faster than that.

From: Python-list  on 
behalf of Dino 
Date: Sunday, January 15, 2023 at 1:29 PM
To: python-list@python.org 
Subject: Re: Fast lookup of bulky "table"
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Thank you for your answer, Lars. Just a clarification: I am already
doing a rough measuring of my queries.

A fresh query without any caching: < 4s.

Cached full query: < 5 micro-s (i.e. 6 orders of magnitude faster)

Desired speed for my POC: 10  Hey,
>
> before you start optimizing. I would suggest, that you measure response
> times and query times, data search times and so on. In order to save
> time, you have to know where you "loose" time.
>
> Does your service really have to load the whole table at once? Yes that
> might lead to quicker response times on requests, but databases are
> often very good with caching themselves, so that the first request might
> be slower than following requests, with similar parameters. Do you use a
> database, or are you reading from a file? Are you maybe looping through
> your whole dataset on every request? Instead of asking for the specific
> data?
>
> Before you start introducing a cache and its added complexity, do you
> really need that cache?
>
> You are talking about saving microseconds, that sounds a bit as if you
> might be “overdoing” it. How many requests will you have in the future?
> At least in which magnitude and how quick do they have to be? You write
> about 1-4 seconds on your laptop. But that does not really tell you that
> much, because most probably the service will run on a server. I am not
> saying that you should get a server or a cloud-instance to test against,
> but to talk with your architect about that.
>
> I totally understand your impulse to appear as good as can be, but you
> have to know where you really need to debug and optimize. It will not be
> advantageous for you, if you start to optimize for optimizing's sake.
> Additionally if you service is a PoC, optimizing now might be not the
> first thing you have to worry about, but about that you made everything
> as simple and readable as possible and that you do not spend too much
> time for just showing how it could work.
>
> But of course, I do not know the tasks given to you and the expectations
> you have to fulfil. All I am trying to say is to reconsider where you
> really could improve and how far you have to improve.
>
>
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!npizb3UAz-jPUnhlimB3_lctLibK5EW4zJwjZVmQ41yV_-2WSm2eQ5cTi8vzOEuCfsdNTjIvIhFcakrX$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!npizb3UAz-jPUnhlimB3_lctLibK5EW4zJwjZVmQ41yV_-2WSm2eQ5cTi8vzOEuCfsdNTjIvIhFcakrX$><https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!npizb3UAz-jPUnhlimB3_lctLibK5EW4zJwjZVmQ41yV_-2WSm2eQ5cTi8vzOEuCfsdNTjIvIhFcakrX$%3chttps:/urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!npizb3UAz-jPUnhlimB3_lctLibK5EW4zJwjZVmQ41yV_-2WSm2eQ5cTi8vzOEuCfsdNTjIvIhFcakrX$%3e>
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!keHKWsb1LGR6u_6BQA04MyEJlnzICq04FNdn8z9BnnjG8NopVu3KiL0k3rMiowxtp87xBUi6OcavBQIqkvzm3bP5$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!keHKWsb1LGR6u_6BQA04MyEJlnzICq04FNdn8z9BnnjG8NopVu3KiL0k3rMiowxtp87xBUi6OcavBQIqkvzm3bP5$>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fast lookup of bulky "table"

[Weatherby,Gerard]

With Postgresql, one can also do pre-processing in Python. 
https://www.postgresql.org/docs/15/plpython.html

While it’s not as convenient to develop as client-side Python, it can be used 
to implement complicated constraints or implement filtering on the server side, 
which reduces the amount of data that has to be sent back to the client.


From: Python-list  on 
behalf of Stefan Ram 
Date: Sunday, January 15, 2023 at 5:03 PM
To: python-list@python.org 
Subject: Re: Fast lookup of bulky "table"
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

dn  writes:
>Some programmers don't realise that SQL can also be used for
>calculations, eg the eponymous COUNT(), which saves (CPU-time and
>coding-effort) over post-processing in Python.

  Yes, I second that! Sometimes, people only re-invent things
  in Python because they don't know SQL well enough, or they
  do not normalize their tables because they have not properly
  learned how to do this.

  I'd always start out with normalized tables and do as many
  operations in SQL as possible. I would then hesitate to
  de-normalize anything or transfer data operations into
  the programming language unless I am very sure that this
  is really advantageous.

  Once I had the task of writing VBA code to query and analyze
  data from a Jet engine (i.e., Microsoft Access). I ended up
  writing 90 % of the code in SQL and a thin layer of 10 % in VBA.
  And it was fast.


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kAIZWRJ3oqrlkixX-iwrGeG9VVWjooBvzuMirfp44VTP32cELWf8Dk6MkPQwK2QwWzuUT9eNPNTlN152b23eFcM$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Improvement to imports, what is a better way ?

[Weatherby,Gerard]

In the body of the code, if every time an external identifier is used it
must be prefixed by a full 'path', the cognitive "burden" shifts from
the aspect highlighted (above), to a reading-burden.

As the fictional Jack McCoy ( https://en.wikipedia.org/wiki/Jack_McCoy) would 
say --

Objection: Assumes facts not in evidence!

If I read A.py:

import …
# many lines of code
x = obscure_module.widget(4)

I know immediately where the function widget is.

If I read B.py
from obscure_module import widget
# many lines of code
x = widget(4)

Now I have to find out where the heck “widget” came from

And C.py

from obscure_module import widget as gadget
# many lines of code
x = gadget(4)

which is actually easier and harder to read.  Personally, I’m much happier with 
A.py than B.py or C.py.







From: Python-list  on 
behalf of dn via Python-list 
Date: Wednesday, January 18, 2023 at 3:49 PM
To: python-list@python.org 
Subject: Re: Improvement to imports, what is a better way ?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 19/01/2023 08.56, Mats Wichmann wrote:
> On 1/18/23 12:29, Paul Bryan wrote:
...

>>> import os asos
>>> import sys as   sys
>>> import importlib as importlib
>
> A general comment: there are some very common "import ... as" idioms
> (for example, it seems like *everyone* does "import pandas as pd") and
> those are okay to follow, but in general I would stay away from trying
> to give everything short-names.  Each module imported with a name other
> than their own is a memory burden for the reader (maybe even for you!).
>
>>> import aboutTime as tt  # Time dates timestamps and the
>>> like
>>> import avMedia as   av  # Audio and maybe video 'someday'
>>> well definitely lots of TTS text to speech
>>> import basicSwitchboard as  sc  # First switchboard lurking.
>>> Kickoff to sequence viewers
>
> Any decent editor these days will autocomplete for you, so there's
> really not much if any typing burden in using the full names.

Have vacillated on aspects of this topic, over the years. Currently I'm
in favor of abbreviations and not only the commonly-accepted/-used ones,
eg np for numpy.
(but am keen to learn from your wisdom)

I've argued the auto-complete cf typing point. So, that not at-issue.

The longer an identifier, the more it 'pushes' code over to the right or
to expand over multiple screen-lines. Some thoughts on this are behind
PEP-008 philosophies, eg line-limit.

In the body of the code, if every time an external identifier is used it
must be prefixed by a full 'path', the cognitive "burden" shifts from
the aspect highlighted (above), to a reading-burden. Thus, (in extreme
cases) moving towards a 'wall of text' problem.

The primary interest is to remove "burden" aka complexity.

In using TDD, code is constructed module-by-module (not necessarily a
Python Module). So, when it comes time to call avMedia.fetch_the_file()
[sic] there is little thinking about the "avMedia" bit. The emphasis is
on the function-name, and it's parameters. 'I need that file so that I
can ...'.

Using the IDE-as-a-tool argument (similar to above): if I miss-out,
mistype, use the wrong abbreviation, or otherwise fail to identify where
fetch_the_file() is located, the IDE will immediately tell me.
(dn you're wrong - again!)

Accordingly, the abbreviation/full-module-name is almost taken-on-trust.
(you're going to prove/test the linkage, either way, right?)


Personal Biases:
- TDD
- when starting to write the program[me]-code one of the first steps is
to marshal resources, which includes the question: which functions will
be called and thus which Python-modules should be imported? Thus,
listing all imports 'at the top' is a separable task.
- an early language learned (back-when) was COBOL, which has a formal
structure and separation between elements/phases of a program[me]'s
construction and execution. Some such thinking no doubt lingers...

--
Regards,
=dn
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nUeLMY3g5EV0oaXoGfknoeDOLU9qMoIxHOe7jhHbHfN1v7-wj17OJCgqL_IyHpAr0EN8gyzj5ZEQZB9_A-rB3A_Q5g$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A natural magnet for the craziest TKinter lovers out there

[Weatherby,Gerard]

Works fine on my work machine. (Ubuntu 20.04 / 32 G / 32 CPUS). Scalene 
(https://github.com/plasma-umass/scalene) shows it using 9 MB of memory.

From: Python-list  on 
behalf of Michael Torrie 
Date: Wednesday, January 18, 2023 at 8:58 PM
To: python-list@python.org 
Subject: Re: A natural magnet for the craziest TKinter lovers out there
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 1/18/23 18:01, Dan Kolis wrote:
> Hangs after maybe between 4 and 50 screen rewrites. sometimes CTRL C under 
> Ubuntu starts it up again. Click go rewrites al the fonts the thing can find 
> in a few windows Repeated.
>

Not sure what you mean by "screen rewrites."

I ran your test program here and it generates 25 windows on my machine,
and I can click "run" at least half a dozen times. I tried closing the
font windows before clicking run again, and also just leaving the
windows up and generating many more windows.  300 windows. No hangs here
at all. Fedora 35 with Mate Desktop on X11 with compositing enabled.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iGQaGdd9t2QPVQJDhyDTQmXgCwx4YG7m8jFbRd11dF1BweJHCd0wC4UyZUfQGQjqxeulMYagnDqInfA81A$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: tree representation of Python data

[Weatherby,Gerard]

https://docs.python.org/3/library/pprint.html

From: Python-list  on 
behalf of Dino 
Date: Saturday, January 21, 2023 at 11:42 AM
To: python-list@python.org 
Subject: tree representation of Python data
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I have a question that is a bit of a shot in the dark. I have this nice
bash utility installed:

$ tree -d unit/
unit/
├── mocks
├── plugins
│   ├── ast
│   ├── editor
│   ├── editor-autosuggest
│   ├── editor-metadata
│   ├── json-schema-validator
│   │   └── test-documents
│   └── validate-semantic
│   ├── 2and3
│   ├── bugs
│   └── oas3
└── standalone
 └── topbar-insert

I just thought that it would be great if there was a Python utility that
visualized a similar graph for nested data structures.
Of course I am aware of indent (json.dumps()) and pprint, and they are
OK options for my need. It's just that the compact, improved
visualization would be nice to have. Not so nice that I would go out of
my way to build, but nice enough to use an exising package.

Thanks

Dino
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iGkQhjN2DTLNPZj7JaAhVrakd6oiQrV3IUV08E2ayIK1hWH2AaJ4OQ_uEobpiLQuWde2974mF41mvsnO$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Weatherby,Gerard]

Argparse is for parsing command line arguments and options.

If you just want to evaluate an Python expression, use eval( )

Your string isn’t valid Python due to order of operations, but 
-(4^2)+5.3*abs(-2-1)/2 is.

From: Python-list  on 
behalf of Jach Feng 
Date: Sunday, January 22, 2023 at 11:24 AM
To: python-list@python.org 
Subject: Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Thomas Passin 在 2023年1月22日 星期日下午1:30:39 [UTC+8] 的信中寫道：
> On 1/21/2023 10:11 PM, Jach Feng wrote:
> > Fail on command line,
> >
> > e:\Works\Python>py infix2postfix.py "-4^2+5.3*abs(-2-1)/2"
> > usage: infix2postfix.py [-h] [infix]
> > infix2postfix.py: error: unrecognized arguments: -4^2+5.3*abs(-2-1)/2
> >
> > Also fail in REPL,
> >
> > e:\Works\Python>py
> > Python 3.8.8 (tags/v3.8.8:024d805, Feb 19 2021, 13:08:11) [MSC v.1928 32 
> > bit (Intel)] on win32
> > Type "help", "copyright", "credits" or "license" for more information.
>  import argparse
>  parser = argparse.ArgumentParser(description='Convert infix notation to 
>  postfix')
>  parser.parse_args("-4^2+5.3*abs(-2-1)/2")
> > usage: [-h]
> > : error: unrecognized arguments: - 4 ^ 2 + 5 . 3 * a b s ( - 2 - 1 ) / 2
> >
> > Just can't figure out where is wrong!?
> It just doesn't work like that. If you download the package, there is
> only one python file, __init__.py. This file contains one class. It
> has a demo at the end, commented out. If you uncomment those lines and
> run the file, you get a result printed.
>
> These remarks are based on downloading the link for the source
> distribution from Pypi
> (https://urldefense.com/v3/__https://pypi.org/project/infix2postfix/__;!!Cn_UX_p3!gqKmYLlyUndAzxmJsqCB429izQ-2-KMbpGP2eVzp_iDKtbgQXfrCu21UBvepq-F9EXb4SJwP516MHeUFMBtW$
>  ). When I installed it with
> pip, nothing seems to have gotten installed although pip went through
> the motions and claimed it was. So I just downloaded the source package.
>
> The test expression is "-(a*b)+(c+d)-(a+b+c+d)". The test output for
> this is "ab*-cd++ab+c+d+-".
>
> If you substitute your expression, the result is
>
> abs1-2-*2/3.5+2^4-
>
> This may or may not be correct. I'm not sure but I think it's as
> intended except for reversing "3.5". But maybe that's right, I'm not
> too sure. Notice that this file is in its first release, version 0.0.1
> - the metadata that says it's 'Development Status :: 5 -
> Production/Stable' seems to be bogus. So it may very well be buggy.
>
> At any rate, if you want to use it in a program that can accept
> arguments, you will have to write that part yourself. And the
> expression you feed it would need to be a single string, meaning it has
> to be quoted on the command line as you have done (although on Windows
> you should be using double quotes instead of single quotes).
>
> As for argparse, it isn't doing what you want because you haven't told
> it what to do with the arguments.
Sorry to cause confusion here. I don't know there is a Pypi project with the 
same name infix2postfix.py:-(

Nevertheless, Is there anyway to make parse_args works?
>>> parser.parse_args("-4^2+5.3*abs(-2-1)/2")
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqKmYLlyUndAzxmJsqCB429izQ-2-KMbpGP2eVzp_iDKtbgQXfrCu21UBvepq-F9EXb4SJwP516MHezHygV-$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? WRONG TOOL

[Weatherby,Gerard]

I understand we all want to be helpful and friendly, but it seems to me that 
helping someone use the wrong tool for the job isn’t really helpful in the long 
run.

argparse is for parsing command line arguments.  It’s the wrong tool for this 
job.

As Chris Angelico already said: This entire thread is a massive "how can I use 
X to do Y?" problem.



From: Python-list  on 
behalf of Thomas Passin 
Date: Tuesday, January 24, 2023 at 10:23 AM
To: Mike Baskin , python-list@python.org 

Subject: Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 1/24/2023 10:13 AM, Mike Baskin wrote:
> Can you stop please

It's way past time, isn't it!

> Sent from Yahoo Mail for iPhone 
>   >
>
> On Tuesday, January 24, 2023, 10:12 AM, Thomas Passin
>  wrote:
>
> On 1/23/2023 9:12 PM, Chris Angelico wrote:
>  > On Tue, 24 Jan 2023 at 13:09, Jach Feng  > wrote:
>  >>
>  >> Chris Angelico 在 2023年1月24日 星期二清晨5:00:27 [UTC+8] 的信中
> 寫道：
>  >>> On Tue, 24 Jan 2023 at 07:47, Cameron Simpson  > wrote:
>  
>   But for Jach Feng: the "--" is really expected as something
> the user
>   does when they invoke your programme, _explicitly_ saying that
> what
>   follows from here is not an argument. So the user is expected
> to type:
>  
>   your_script -x -y -- "-4^2+5.3*abs(-2-1)/2"
>  
>   where there are -x and -y options, then end of options, then an
>   argument, which would look like an option if there wasn't the "--"
>   argument.
>  >>> And if you DON'T expect the user to enter the "--", then why use
>  >>> argparse? You can just check argv directly to get your arguments.
>  >>>
>  >>> This entire thread is a massive "how can I use X to do Y?" problem.
>  >>>
>  >>> ChrisA
>  >> The '--' requirement makes its usage less instinctive, and
> handling argv directly makes me loss the benefit of using '-h':-)
>  >
>  > if "-h" in sys.argv: usage()
>  > else: do_stuff_with(sys.argv[1:])
>  >
>  > What is argparse really doing for you?
>
> I second this.  "if '-h' in sys.argv:"  is usually what I do.
>
> Alternatively, you could use "--arg=" syntax and place your string
> "-4^2+5.3*abs(-2-1)/2" its right-hand side":
>
> infix2postfix [options] "--infix=-4^2+5.3*abs(-2-1)/2"
>
> This shouldn't be too hard for a user to work with.  You could scan the
> argument list for the presence of "--infix=" and display the help
> message if it isn't there.
>
>
> --
> 
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!g3YRQIW4wminyTVpjV9kATW-QRZ1YhvDi82oJvatyhNe_OABSPTU9c8KIOlztBNofs690OPwUdIykR5ILj4$
> 
>   >
>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!g3YRQIW4wminyTVpjV9kATW-QRZ1YhvDi82oJvatyhNe_OABSPTU9c8KIOlztBNofs690OPwUdIykR5ILj4$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: bool and int

[Weatherby,Gerard]

Booleans work exactly the way the language documentation says they work:

Booleans (bool)
These represent the truth values False and True. The two objects representing 
the values False and True are the only Boolean objects. The Boolean type is a 
subtype of the integer type, and Boolean values behave like the values 0 and 1, 
respectively, in almost all contexts, the exception being that when converted 
to a string, the strings "False" or "True"are returned, respectively.

https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarchy

From: Python-list  on 
behalf of Dino 
Date: Tuesday, January 24, 2023 at 3:04 PM
To: python-list@python.org 
Subject: bool and int
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

$ python
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
 >>> b = True
 >>> isinstance(b,bool)
True
 >>> isinstance(b,int)
True
 >>>

WTF!

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jPbvUX9ZXFGYt-q850YI6aQ7ET7BwbF-LIT4XT7MKKwF9OSOgqnaHdM4MbQ7p6YaRCYJYXZ4-XiT3Sko$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: bool and int

[Weatherby,Gerard]

https://peps.python.org/pep-0285/

From: Python-list  on 
behalf of rbowman 
Date: Tuesday, January 24, 2023 at 3:01 PM
To: python-list@python.org 
Subject: Re: bool and int


bool is a subtype of integer. I never dug that deep into Python's guts but
I assume it goes back to boolean being an afterthought in C. Some people
fancy it up with #defines but I always use int.  0 is false, anything else
is true.

C# is pickier, which I guess is a good thing.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hdxuMNsprOXvH5ouxGfbGLLq6wuXs-_gOESRVYUxDsHYCmlrpv9ru-WYMziYU4FRdum02bS6DfRnNDnCNQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

[Weatherby,Gerard]

Use a different prefix character

parser = argparse.ArgumentParser(prefix_chars='%')
parser.add_argument('expression')

args = parser.parse_args()
print(args.expression)

argparser is for allowing multiple command line options to be passed, providing 
default, controlling the number of arguments and the like.


From: Python-list  on 
behalf of Jach Feng 
Date: Wednesday, January 25, 2023 at 12:25 PM
To: python-list@python.org 
Subject: Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Chris Angelico 在 2023年1月25日 星期三下午1:16:25 [UTC+8] 的信中寫道：
> On Wed, 25 Jan 2023 at 14:42, Jach Feng  wrote:
> > I was happy working with argparse during implement my script. To save the 
> > typing, I used a default equation for testing.
> >
> > sample = "-4^2+5.3*abs(-2-1)/2, abs(Abc)*(B+C)/D, (-3) * 
> > sqrt(1-(x1/7)*(y1/7)) * sqrt(abs((x0-4.5)/(y0-4)))"
> > parser = argparse.ArgumentParser(description='Convert infix notation to 
> > postfix')
> > parser.add_argument('infix', nargs='?', default=sample, help="")
> >
> You're still not really using argparse as an argument parser. Why not
> just do your own -h checking? Stop trying to use argparse for what
> it's not designed for, and then wondering why it isn't doing what you
> expect it to magically know.
>
> ChrisA
I just don't get what you mean?

> You're still not really using argparse as an argument parser. Why not just do 
> your own -h checking?

Is a math equation not qualified as a command line "argument"? What criteria do 
you use when judging the quality of an "argument"?

> Stop trying to use argparse for what it's not designed for,

Even the author considers a positional argument begin with '-' is a legal 
argument. Below is a quote from its manual.

"If you have positional arguments that must begin with - and don’t look like 
negative numbers, you can insert the pseudo-argument '--' which tells 
parse_args() that everything after that is a positional argument"

> and then wondering why it isn't doing what you expect it to magically know."

I don't expect magic, I expect the consistency of a parser.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jfetVCL_U0v2RwXvkQWkim9VOYbZSJsWxRjWVlhpCDNHLoBWGyPJKGC_vh1Hwkrm3AzcX2KNEOUFKNNBx0tG$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: bool and int

[Weatherby,Gerard]

I can’t help but wonder if there exists some Java forum /mailing list going on 
about how horrible Python is.

From: Python-list  on 
behalf of rbowman 
Date: Wednesday, January 25, 2023 at 12:25 PM
To: python-list@python.org 
Subject: Re: bool and int
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On Wed, 25 Jan 2023 06:53:44 -0500, 2QdxY4RzWzUUiLuE wrote:


> They used Java at my last job (as in, the last job I had before I
> retired), and it was absolutely awful, for any number of reasons, the
> gymnastics (on many levels) required to support "primitive types" being
> one of them.

My first brush with Java was around '98 when it was first becoming
popular. To familiarize myself with the AWT I decided to write a simple
IDE for the AVR microcontrollers. What a disaster. The UI wasn't bad but
the instructions for 8-bit processors require a lot of bit fiddling that
was extraordinarily difficult in Java.

Then they came out with Swing and the assumption if the app ran with
glacial slowness you should get a faster machine.

The company I work for has one Java app created around 2000 as a cross
platform solution as people moved to Windows. Originally it ran as an
applet but when that window was slammed shut it became increasingly
unwieldy.

For what I'm developing today I used either .NET C# or Python3. The .NET
UI's on Linux aren't quite there yet but back end applications are fine.
PyQt (PySide actually. If there is a way to screw up commercial licensing
Qt will find it) is fine.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!iVrdROvxcoNV-GhzezJe8fJSLSAUoPkZHaXF58tWtyogy37PB6b9DH-gINgbVLuU64V4RovArDpnC5jjiQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Custom help format for a choice argparse argument

[Weatherby,Gerard]

Why not something like:

parser.add_argument("-z", "--zone")
   args = parser.parse_args()
   if args.zone and args.zone not in ptyz.all_timezones:
print(“Invalid timezone”,file=sys.stderr)
…



From: Python-list  on 
behalf of Ivan "Rambius" Ivanov 
Date: Friday, January 27, 2023 at 3:33 PM
To: Python 
Subject: Custom help format for a choice argparse argument
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hello,

I am developing a script that accepts a time zone as an option. The
time zone can be any from pytz.all_timezones. I have

def main():
parser = argparse.ArgumentParser()
parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
args = parser.parse_args()
print(args)
print(f"Specified timezone: {args.zone}")

It works, but when I run it with the -h option it dumps all entries in
pytz.all_timezones. I would like to modify the help format for just
-z|--zone option. I read the docs about HelpFormatter and argparse.py
and I ended up with

class CustomHelpFormatter(argparse.HelpFormatter):
def _metavar_formatter(self, action, default_metavar):
if action.dest == 'zone':
result = 'zone from pytz.all_timezones'
def format(tuple_size):
if isinstance(result, tuple):
return result
else:
return (result, ) * tuple_size
return format
else:
return super(CustomHelpFormatter,
self)._metavar_formatter(action, default_metavar)


def main():
parser = argparse.ArgumentParser(formatter_class=CustomHelpFormatter)
parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
args = parser.parse_args()
print(args)
print(f"Specified timezone: {args.zone}")

This works, but is there a more elegant way to achieve it?

Regards
rambius

--
Tangra Mega Rock: 
https://urldefense.com/v3/__http://www.radiotangra.com__;!!Cn_UX_p3!kiJusdm5pCptP3sOBX85KXqUJkqr2jSa4C_-WAqND7WkL-aw3BYbW50td_AcuzJ1XUPYIVO3JiLMc4gRWS885vTKFsFvaQ$
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kiJusdm5pCptP3sOBX85KXqUJkqr2jSa4C_-WAqND7WkL-aw3BYbW50td_AcuzJ1XUPYIVO3JiLMc4gRWS885vRXq-JKLg$
-- 
https://mail.python.org/mailman/listinfo/python-list

Unix / linux programs

[Weatherby,Gerard]

The Unix convention is 0 means everything went well, and non-zero means 
something else happened. Here’s a contrived example of a bash wrapper around 
GNU tar. By contrived I mean it works but I would not use it in practice … I’d 
just use tar directly or use the Python tarfile module if I wanted finer grain 
control of the process.

#!/bin/bash
if [ $# -lt 2 ]; then
echo "Usage: $0 [tar] [directory to compare]"
fi
tar --diff -f $1 $2 2>/dev/null
r=$?
if [ $r -eq 0 ]; then
echo $1 has $2 in it, unmodified
elif [ $r -eq 1 ]; then
echo $1 does not have $2 in it unmodified
elif [ $r -eq 2 ]; then
# maybe do more tests here?
echo There is a problem with $1 or with $2
else
echo "Other error $r"
fi

Incidentally, I found the man page installed on Ubuntu20.04 for tar is out of 
date. tar returns 64 if doesn’t like the command line arguments. Nothing works 
until it is tested.

-- 
https://mail.python.org/mailman/listinfo/python-list

Non int Boolean

[Weatherby,Gerard]

If someone really really wants a non-int Boolean, it is easy to implement. 5 or 
6 lines, depending on whether you count the import statement:

from enum import Enum


class MyBool(Enum):
TRUE = 42
FALSE = 54


def __bool__(self):
return self == MyBool.TRUE


#
# testing
#
mytrue = MyBool.TRUE
try:
print(int(mytrue)) #this fails
except TypeError as te:
print(te)

asbool = bool(mytrue)
if mytrue:
print("yep")

myfalse = MyBool.FALSE
if myfalse:
print("nope")

---
I would never use such a thing, and I would be annoyed if I can across code 
that did. As has been said (beaten to death?) Python has an existing 
well-understood boolean type.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: evaluation question

[Weatherby,Gerard]

import io

def countprint(*args, **kwargs):
capturekw = {k:v for k,v in kwargs.items() if k != 'file'}
buffer = io.StringIO()
capturekw['file'] = buffer
print(*args,**kwargs)
print(*args,**capturekw)
return len(buffer.getvalue())

def boolprint(*args,active:bool, **kwargs):
if active:
print(*args,**kwargs)

with open("text.txt",'w') as f:
y  = countprint(1, 3, 3, sep=',', end='\n\n',file=f)
print(y)
boolprint(3,4,5,sep='/',active=True)
boolprint(7,11,active=False)

From: Python-list  on 
behalf of avi.e.gr...@gmail.com 
Date: Tuesday, January 31, 2023 at 3:01 PM
To: 'Thomas Passin' , python-list@python.org 

Subject: RE: evaluation question
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I think its has been discussed here that many functions are DELIBERATELY
designed to return without returning anything. Earlier languages like Pascal
had explicit ideas that a function that did not return a value was declared
as a "procedure" but many other languages like python make no real
differentiation.

Some functions are designed for a sort of side-effect and often there is
nothing much that needs to be returned or even can be. If a function prints
a dozen items one at a time, should it return nothing, or a copy of the last
item or somehow of all items? Generally nothing looks right. If you want to
return something, fine. Do it explicitly.

Similar arguments have been made about methods that do things like sort the
contents of an object internally and then return nothing. Some would like
the return to be the (now altered) object itself. You can emulate that by
not sorting internally but instead sorted(object) returns a new object that
has been sorted from the old one.

So should or could print return anything? Other languages exist, like R,
that do return (and often ignore) whatever print displayed elsewhere. This
can be of use in many ways such as making it easier to print or store
additional copies without recalculating.

My preference might be to simply allow a local option at the end of a print
statement such as print(..., return=True) or even a way to set a global
option so all print statements can be turned on when you want. But is this
pythonic? In particular, people who want to give type hints now can safely
claim it returns None and would have to modify that so it can optionally
return something like str or None. And, of course, once you change print()
this way, someone else will want the number of characters (or perhaps bytes)
returned instead.

Much of this can be worked around by simply making your own customized print
function which evaluates the arguments to make a string and then calls
print, perhaps with the results pre-calculated, and returns what you wanted.
That is not as easy as it sounds, though as print  supports various
arguments like sep= and end= and file= and flush= so a weird but doable idea
is simply to substitute a temporary file for any file= argument and write
the results to a temporary file or something in memory that emulates a file.
You can then read that back in and return what you want after handling the
original print statement with the original arguments, or perhaps just use
your result to any actually specified file or the default.

You can thus create something like what you want and leave the original
print() command alone to do what it was designed to do.

And, in general, people who want a copy of what they print, often use other
python functionality to craft some or all parts of the text they want
printed and only then call print() and thus already may have the ability to
use the text afterwards.

For many purposes, including efficiency, returning nothing makes good sense.
But it is not really the only choice or the right choice and yet, if you
want to use THIS language, it has to be accepted as the documented choice.


-Original Message-
From: Python-list  On
Behalf Of Thomas Passin
Sent: Tuesday, January 31, 2023 1:16 PM
To: python-list@python.org
Subject: Re: evaluation question

On 1/31/2023 4:24 AM, mutt...@dastardlyhq.com wrote:
> On Tue, 31 Jan 2023 12:57:33 +1300
> Greg Ewing  wrote:
>> On 30/01/23 10:41 pm, mutt...@dastardlyhq.com wrote:
>>> What was the point of the upheaval of converting the print command
>>> in python 2 into a function in python 3 if as a function
>>> print() doesn't return anything useful?
>>
>> It was made a function because there's no good reason for it to have
>> special syntax in the language.
>
> All languages have their ugly corners due to initial design mistakes
> and/or constraints. Eg: java with the special behaviour of its string
> class, C++ with "=0" pure virtual declaration. But they don't dump
> them and make all old code suddenly cease to execute.
>
> Pragmatism should always come before language purity.
>

It was more fundamental than that, and not mainly about print():

https://urldefense.com/v3/__https://snarky.ca

Re: Organizing modules and their code

[Weatherby,Gerard]

You’re overthinking it. It doesn’t really matter. Having small chunks of codes 
in separate files can be hassle when trying to find out what the program does. 
Having one file with 2,000 lines in it can be a hassle.  This is art / opinion, 
not science.

From: Python-list  on 
behalf of transreductionist 
Date: Friday, February 3, 2023 at 4:48 PM
To: python-list@python.org 
Subject: Organizing modules and their code
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Here is the situation. There is a top-level module (see designs below) 
containing code, that as the name suggests, manages an ETL pipeline. A 
directory is created called etl_helpers that organizes several modules 
responsible for making up the pipeline. The discussion concerns the Python 
language, which supports OOP as well as Structural/Functional approaches to 
programming.

I am interested in opinions on which design adheres best to standard 
architectural practices and the SOLID principles. I understand that this is one 
of those topics where people may have strong opinions one way or the other. I 
am interested in those opinions.

Allow me to give my thoughts. First, I don't think there would be much 
difference if I was using OOP for the functionality, or using a structural 
paradigm. A structural paradigm in my opinion, along the lines of Rich Hickey's 
comments on simple versus complex, would be a simpler implementation. In this 
case there is no reason to create a construct with state. So let's assume the 
code is structural and not OOP.

I would go with Design I. Succinctly stated, Design I supports readability and 
maintainability at least as well, if not better than the other designs. The 
goal of the SOLID principles are the creation of mid-level software structures 
that (Software Architecture: SA Martin). I think Design I best adheres to these 
principles of:
 Tolerate change,
 Are easy to understand, and
 Are the basis of components that can be used in many software systems.

I could point to the Single Responsibility Principle which is defined as (SA 
Martin): a module should be responsible to one, and only one, actor. It should 
satisfy the Liskov Substitution Principle as well. Further, each module in the 
etl_helpers directory is at the same level of abstraction.

I could also mention that as Dijkstra stressed, at every level, from the 
smallest function to the largest component, software is like a science and, 
therefore, is driven by falsifiability. Software architects strive to define 
modules, components, and services that are easily falsifiable (testable). To do 
so, they employ restrictive disciplines similar to structured programming,
albeit at a much higher level (SA Martin).

One can point to multiple reasons why Design I might be preferred, but what are 
the compelling reasons, if there are any, that would suggest another design was 
superior.

Finally, let me reference an interesting research paper I read recently that 
seems to support the other designs as anti-patterns: 
Architecture_Anti-patterns_Automatically.pdf

  
(https://urldefense.com/v3/__https://www.cs.drexel.edu/*yfcai/papers/2019/tse2019.pdf__;fg!!Cn_UX_p3!jcpCdxiLoPobR0IdlyJHwyPiNP4_iVC6dAMtg_HsLr5hStszx-WnYyZQHJ-4pJTOGsw4-6pEGJyDpSytZQqfpvATg06FMA$
 )

SEVERAL DESIGNS FOR COMPARISON

DESIGN I:

 manage_the_etl_pipeline.py
 etl_helpers
   extract.py
   transform.py
   load.py

Of course one could also

DESIGN II:

 manage_the_etl_pipeline.py
 etl_helpers
   extract_transform_load.py

or probably even:

DESIGN III:

 manage_the_etl_pipeline.py
 extract_transform_load.py
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jcpCdxiLoPobR0IdlyJHwyPiNP4_iVC6dAMtg_HsLr5hStszx-WnYyZQHJ-4pJTOGsw4-6pEGJyDpSytZQqfpvBaJ2e2VA$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Typing Number, PyCharm

[Weatherby,Gerard]

dn,

I’m missing something here. Method 5 seems to work fine in PyCharm. I’m 
interpreting your statement as:

from fractions import Fraction
from numbers import Number


def double(value: Number):
if isinstance(value, Number):
# noinspection PyTypeChecker
return 2 * value
raise ValueError(f"{value} of {type(value)} is not a Number")


print(double(7))
print(double(7.2))
print(double(complex(3.2, 4.5)))
print(double(Fraction(7, 8)))
# print(double("7")) PyCharm properly complains


From: Python-list  on 
behalf of dn via Python-list 
Date: Saturday, February 4, 2023 at 9:32 PM
To: 'Python' 
Subject: Typing Number, PyCharm
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Do we have a typing type-hint for numbers yet?


Often wanting to combine int and float, discovered that an application
was doing a walk-through with/for uses three numeric types. Was
intrigued to note variance, in that the code-set features two different
methods for typing, in this situation:

def func( value ):
 ...using value...

where value may be an integer, a floating-point value, or a
complex-number (but not decimal-type).
NB code snippets from memory (cf copy-paste)


Method 1 (possibly older code):-

from typing import Union
...
def fun( value:Union[ int, float, complex ] ):


Method 2:-

def fun( value:int|float|complex  ):


Pondering this, realised could use an alias to de-clutter the
function-definition/signature:

Method 3:-

number_type = int|float|complex
...
def fun( value:number_type  ):


If it was important to have type consistency within the union, eg
argument and return, could go for:

Method 4:-

from typing import TypeVar
number_type = TypeVar( 'number_type', int, float, complex )
...
def fun( value:number_type  ):


Then remembered the way we'd code an execution-time check for this using
isinstance():

Method 5:-

from numbers import Number
...
def fun( value:Number  ):


Each of these will execute correctly.

All cause PyCharm to object if I try to call the fun(ction) with a
string parameter - and execute an exception, as expected.


Accepting all the others, am curious as to why PyCharm objects to Method
5 with "Expected type 'SupportsFloat | SupportsComplex | complex |
SupportsIndex', got 'Number' instead? - yet still highlights the
erroneous string parameter but none of the 'legal' data-types?

As soon as a list (in this case types) reaches three, my aged-eyes start
to think de-cluttering is a good idea!

Do you know of another way to attack this/more properly?

--
Regards,
=dn
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqIbcK1zaXmlo5y6741fRwcBUcDfxPNkiA4Jy_NHr9nEno2HaBGZYMuitXeivWrGwTJtds01pHdFfiY_Y5bnmsq_NQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Organizing modules and their code

[Weatherby,Gerard]

Well, first of all, while there is no doubt as to Dijkstra’s contribution to 
computer science, I don’t think his description of scientific thought is 
correct. The acceptance of Einstein’s theory of relativity has nothing to do 
with internal consistency or how easy or difficult to explain but rather 
repeatedly experimental results validating it. Or, more precisely, not 
disproving it. See Feynmann: https://www.youtube.com/watch?v=0KmimDq4cSU


Engineering is simply maximizing the ratio: benefit / cost. Highly recommend To 
Engineer is Human  by Henry Petroski.

Regarding the initial question: none of the suggested designs would work 
because they lack __init__.py file.

Once the __init__.py is added, the construct of the import statements within it 
will determine how the API looks. All three of Design I / Design II and Design 
III can be implemented with the same API. (I’m pretty sure that’s true. If it’s 
not, I’d be interested in a counterexample).





From: Python-list  on 
behalf of transreductionist 
Date: Saturday, February 4, 2023 at 7:42 PM
To: python-list@python.org 
Subject: Re: Organizing modules and their code
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Thank you for all the helpful replies and consideration. I do hope for other 
opinions

I would rather say it is more like engineering than art. Whether it is a matter 
of overthinking, or under thinking, is another matter. I enjoyed Dijkstra's 
letter to his colleagues on the role of scientific thought in computer 
programming. It is located at:

 
https://urldefense.com/v3/__https://www.cs.utexas.edu/users/EWD/transcriptions/EWD04xx/EWD447.html__;!!Cn_UX_p3!nME8OhiOxAzmzM3jzg6uXZU851dhWWD9JGB8ZRZIzyUzGkmCN-C6SSXrL59eA2KVIh-y-W0VycJSNb8aYcNnc3jd5Pi2fw$

It is my academic training in physics that makes me enjoy picking up an idea 
and examining  it from all sides, and sharing thoughts with friends. Just 
inquisitive curiosity, and not a homework problem,. Thanks for the great link 
to the ETL site. That was a good read. A few years back I built a prod ETL 
application in Golang using gRPC with a multiprocessing pipeline throughout. It 
handled GB of data and was fast.

This analogy came to me the other day. For me, I would rather walk into a 
grocery store where the bananas, apples, and oranges are separated in to their 
own bins, instead of one common crate.


On Friday, February 3, 2023 at 4:18:57 PM UTC-5, transreductionist wrote:
> Here is the situation. There is a top-level module (see designs below) 
> containing code, that as the name suggests, manages an ETL pipeline. A 
> directory is created called etl_helpers that organizes several modules 
> responsible for making up the pipeline. The discussion concerns the Python 
> language, which supports OOP as well as Structural/Functional approaches to 
> programming.
>
> I am interested in opinions on which design adheres best to standard 
> architectural practices and the SOLID principles. I understand that this is 
> one of those topics where people may have strong opinions one way or the 
> other. I am interested in those opinions.
>
> Allow me to give my thoughts. First, I don't think there would be much 
> difference if I was using OOP for the functionality, or using a structural 
> paradigm. A structural paradigm in my opinion, along the lines of Rich 
> Hickey's comments on simple versus complex, would be a simpler 
> implementation. In this case there is no reason to create a construct with 
> state. So let's assume the code is structural and not OOP.
>
> I would go with Design I. Succinctly stated, Design I supports readability 
> and maintainability at least as well, if not better than the other designs. 
> The goal of the SOLID principles are the creation of mid-level software 
> structures that (Software Architecture: SA Martin). I think Design I best 
> adheres to these principles of:
>  Tolerate change,
>  Are easy to understand, and
>  Are the basis of components that can be used in many software systems.
>
> I could point to the Single Responsibility Principle which is defined as (SA 
> Martin): a module should be responsible to one, and only one, actor. It 
> should satisfy the Liskov Substitution Principle as well. Further, each 
> module in the etl_helpers directory is at the same level of abstraction.
>
> I could also mention that as Dijkstra stressed, at every level, from the 
> smallest function to the largest component, software is like a science and, 
> therefore, is driven by falsifiability. Software architects strive to define 
> modules, components, and services that are easily falsifiable (testable). To 
> do so, they employ restrictive disciplines similar to structured p

Re: Typing Number, PyCharm

[Weatherby,Gerard]

“is numbers.Number the preferred type-hint when multiple numeric types are to 
be accepted?”

I don’t know.

On the one hand, it is a well-known type, so it should be recognizable to users 
of an API. On the other hand, Number is entirely abstract, so it doesn’t 
provide useful type checking for the implementation; I had to add # 
noinspection PyTypeChecker to 2 * value to keep PyCharm from complaining. 
Additionally, it does not include the Decimal type.  Interestingly, It was 
added in 2007 in anticipation of “if and when overloading based on types is 
added to the language.” This now seems unlikely to happen. ( 
https://peps.python.org/pep-3141/#rationale )

On the other hand, a specific type alias is clearer to type checkers and 
possibly to the user of an API?

Experimenting, it appears isinstance checks of Type Aliases don’t work before 
3.10, and its TypeAlias construct. That is:

Numeric = Union[int, float, complex, Fraction]


def double(value: Numeric):
if isinstance(value, Numeric):
return 2 * value
raise ValueError(f"{value} of {type(value)} is not Numeric")
Fails but

Numeric : TypeAlias = Union[int, float, complex, Fraction]


def double(value: Numeric):
if isinstance(value, Numeric):
return 2 * value
raise ValueError(f"{value} of {type(value)} is not Numeric")

works (>= 3.10)


From: dn 
Date: Sunday, February 5, 2023 at 2:54 PM
To: Weatherby,Gerard , 'Python' 
Subject: Re: Typing Number, PyCharm
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

No @Gerard, YOU weren't missing anything: since posting, have upgraded
PyCharm to 2022.3.2 and the complaints about 'Method 5' have
disappeared. Evidently a PyCharm issue!

Which alters the top-line question to: is numbers.Number the preferred
type-hint when multiple numeric types are to be accepted?

PS no Decimal(s) nor Fraction(s) in my situation, but may be worth
adding to a wider discussion...


On 06/02/2023 04.03, Weatherby,Gerard wrote:
> dn,
>
> I’m missing something here. Method 5 seems to work fine in PyCharm. I’m
> interpreting your statement as:
>
> from fractions import Fraction
> from numbers import Number
>
>
> def double(value: Number):
> if isinstance(value, Number):
> /# noinspection PyTypeChecker
> /return 2 * value
> raise ValueError(f"{value}of {type(value)}is not a Number")
>
>
> print(double(7))
> print(double(7.2))
> print(double(complex(3.2, 4.5)))
> print(double(Fraction(7, 8)))
> /# print(double("7")) PyCharm properly complains/
>
> *From: *Python-list 
> on behalf of dn via Python-list 
> *Date: *Saturday, February 4, 2023 at 9:32 PM
> *To: *'Python' 
> *Subject: *Typing Number, PyCharm
>
> *** Attention: This is an external email. Use caution responding,
> opening attachments or clicking on links. ***
>
> Do we have a typing type-hint for numbers yet?
>
>
> Often wanting to combine int and float, discovered that an application
> was doing a walk-through with/for uses three numeric types. Was
> intrigued to note variance, in that the code-set features two different
> methods for typing, in this situation:
>
> def func( value ):
>   ...using value...
>
> where value may be an integer, a floating-point value, or a
> complex-number (but not decimal-type).
> NB code snippets from memory (cf copy-paste)
>
>
> Method 1 (possibly older code):-
>
> from typing import Union
> ...
> def fun( value:Union[ int, float, complex ] ):
>
>
> Method 2:-
>
> def fun( value:int|float|complex  ):
>
>
> Pondering this, realised could use an alias to de-clutter the
> function-definition/signature:
>
> Method 3:-
>
> number_type = int|float|complex
> ...
> def fun( value:number_type  ):
>
>
> If it was important to have type consistency within the union, eg
> argument and return, could go for:
>
> Method 4:-
>
> from typing import TypeVar
> number_type = TypeVar( 'number_type', int, float, complex )
> ...
> def fun( value:number_type  ):
>
>
> Then remembered the way we'd code an execution-time check for this using
> isinstance():
>
> Method 5:-
>
> from numbers import Number
> ...
> def fun( value:Number  ):
>
>
> Each of these will execute correctly.
>
> All cause PyCharm to object if I try to call the fun(ction) with a
> string parameter - and execute an exception, as expected.
>
>
> Accepting all the others, am curious as to why PyCharm objects to Method
> 5 with "Expected type 'SupportsFloat | SupportsComplex | complex |
> SupportsIndex', got 'Number' instead? - yet still highlights the
> erroneous string parameter but none of 

Re: Typing Number, PyCharm

[Weatherby,Gerard]

Yeah, I’m confused.
“The Decimal Type<https://peps.python.org/pep-3141/#the-decimal-type>

After consultation with its authors it has been decided that the Decimal type 
should not at this time be made part of the numeric tower.”  
https://peps.python.org/pep-3141/#the-decimal-type

And the Decimal definition I’m finding says:
class Decimal(object):

yet

print(issubclass(Decimal,Number))

returns True.




From: Paul Bryan 
Date: Monday, February 6, 2023 at 9:25 AM
To: Weatherby,Gerard , dn 
, 'Python' 
Subject: Re: Typing Number, PyCharm
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***
On Mon, 2023-02-06 at 12:11 +, Weatherby,Gerard wrote:

On the one hand, it is a well-known type, so it should be recognizable to users 
of an API. On the other hand, Number is entirely abstract, so it doesn’t 
provide useful type checking for the implementation; I had to add # 
noinspection PyTypeChecker to 2 * value to keep PyCharm from complaining. 
Additionally, it does not include the Decimal type.

Hmm...

Python 3.10.9 (main, Dec 19 2022, 17:35:49) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from numbers import Number
>>> from decimal import Decimal as D
>>> isinstance(D("1.0"), Number)
True

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Tool that can document private inner class?

[Weatherby,Gerard]

Yes.

Inspect module

import inspect


class Mine:

def __init__(self):
self.__value = 7

def __getvalue(self):
"""Gets seven"""
return self.__value


mine = Mine()
data = inspect.getdoc(mine)
for m in inspect.getmembers(mine):
if '__getvalue' in m[0]:
d = inspect.getdoc(m[1])
print(d)


From: Python-list  on 
behalf of Ian Pilcher 
Date: Tuesday, February 7, 2023 at 3:34 PM
To: python-list@python.org 
Subject: Tool that can document private inner class?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

I've been banging my head on Sphinx for a couple of days now, trying to
get it to include the docstrings of a private (name starts with two
underscores) inner class.  All I've managed to do is convince myself
that it really can't do it.

See 
https://urldefense.com/v3/__https://github.com/sphinx-doc/sphinx/issues/11181__;!!Cn_UX_p3!nTY5lSlEetgdaUdvelXo0VuYFt8-2nTnGIxK5whhkN8wXilJfeWOjyS9RJfSLbu9R6cwmmtjxYYDCfh9C4AH_g$
 .

Is there a tool out there that can do this?

--

Google  Where SkyNet meets Idiocracy

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nTY5lSlEetgdaUdvelXo0VuYFt8-2nTnGIxK5whhkN8wXilJfeWOjyS9RJfSLbu9R6cwmmtjxYYDCfh_9Xz7jg$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Tool that can document private inner class?

[Weatherby,Gerard]

No.

I interpreted your query as “is there something that can read docstrings of 
dunder methods?”

Have you tried the Sphinx specific support forums? 
https://www.sphinx-doc.org/en/master/support.html

From: Ian Pilcher 
Date: Tuesday, February 7, 2023 at 4:01 PM
To: Weatherby,Gerard , python-list@python.org 

Subject: Re: Tool that can document private inner class?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2/7/23 14:53, Weatherby,Gerard wrote:
> Yes.
>
> Inspect module
>
> import inspect
>
>
> class Mine:
>
> def __init__(self):
> self.__value = 7
>
> def __getvalue(self):
> /"""Gets seven"""
> /return self.__value
>
>
> mine = Mine()
> data = inspect.getdoc(mine)
> for m in inspect.getmembers(mine):
> if '__getvalue' in m[0]:
>  d = inspect.getdoc(m[1])
> print(d)
>

Can inspect generate HTML documentation, à la Sphinx and other tools?

--

Google  Where SkyNet meets Idiocracy

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: evaluation question

[Weatherby,Gerard]

“Why are we even still talking about this?”

Because humans are social creatures and some contributors to the list like to 
discuss things in depth.


From: Python-list  on 
behalf of avi.e.gr...@gmail.com 
Date: Friday, February 10, 2023 at 6:19 PM
To: python-list@python.org 
Subject: RE: evaluation question
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

There are no doubt many situations someone wants to know how long something
will be when printed but often at lower levels.

In variable-width fonts, for example, the number of characters does not
really line up precisely with how many characters. Some encodings use a
varying number of bytes and, again, the width of the output varies.

So for people who want to make 2-D formatted output like tables, or who want
to wrap lines longer than N characters, you more often let some deeper
software accept your data and decide on formatting it internally and either
print it at once, when done calculating, or in the case of some old-style
terminals, use something like the curses package that may use escape
sequences to update the screen more efficiently in various ways.

If someone wants more control over what they print, rather than asking the
print() function to return something that is mostly going to be ignored,
they can do the things others have already suggested here. You can make your
message parts in advance and measure their length or anything else before
you print. Or make a wrapper that does something for you before calling
print, perhaps only for common cases and then returns the length to you
after printing.

I wonder if the next request will be for  print() to know what your output
device is and other current settings so it return the width your text takes
up in pixels in the current font/size ...

I add a tidbit that many ways of printing allow you to specify the width you
want something printed in such as you want a floating point value with so
many digits after the decimal point in a zero or space padded field on the
left. So there are ways to calculate in advance for many common cases as to
how long each part will be if you specify it. Besides, I am not really sure
if "print" even knows easily how many characters it is putting out as it
chews away on the many things in your request and calls dunder methods in
objects so they display themselves and so on. I assume it can be made to
keep track, albeit I can imagine printing out an APL program with lots of
overwritten characters where the number of bytes sent is way more than the
number of spaces in the output.

Why are we even still talking about this? The answer to the question of why
print() does not return anything, let alone the number of characters
printed, is BECAUSE.


-Original Message-
From: Python-list  On
Behalf Of Python
Sent: Friday, February 10, 2023 4:56 PM
To: python-list@python.org
Subject: Re: evaluation question

On Sat, Feb 11, 2023 at 08:30:22AM +1100, Chris Angelico wrote:
> On Sat, 11 Feb 2023 at 07:36, Python  wrote:
> > You would do this instead:
> >
> > message = f"{username} has the occupation {job}."
> > message_length = len(message)
> > print(message)
> > print(message_length)
> > ...
> >
>
> It's worth noting WHY output functions often return a byte count. It's
> primarily for use with nonblocking I/O, with something like this:
>
> buffer = b".."
> buffer = buffer[os.write(fd, buffer):]
>
> It's extremely important to be able to do this sort of thing, but not
> with the print function, which has a quite different job.

I would agree with this only partially.  Your case applies to os.write(),
which is essentially just a wrapper around the write() system call, which
has that sort of property... though it applies also to I/O in blocking mode,
particularly on network sockets, where the number of bytes you asked to
write (or read) may not all have been transferred, necessitating trying in a
loop.

However, Python's print() function is more analogous to C's printf(), which
returns the number of characters converted for an entirely different
reason... It's precisely so that you'll know what the length of the string
that was converted is.  This is most useful with the
*snprintf() variants where you're actually concerned about overrunning the
buffer you've provided for the output string, so you can realloc() the
buffer if it was indeed too small, but it is also useful in the context of,
say, a routine to format text according to the size of your terminal.  In
that context it really has nothing to do with blocking I/O or socket
behavior.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$

Re: Precision Tail-off?

[Weatherby,Gerard]

Use Python3

Use the decimal module:  https://docs.python.org/3/library/decimal.html


From: Python-list  on 
behalf of Stephen Tucker 
Date: Tuesday, February 14, 2023 at 2:11 AM
To: Python 
Subject: Precision Tail-off?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hi,

I have just produced the following log in IDLE (admittedly, in Python
2.7.10 and, yes I know that it has been superseded).

It appears to show a precision tail-off as the supplied float gets bigger.

I have two questions:
1. Is there a straightforward explanation for this or is it a bug?
2. Is the same behaviour exhibited in Python 3.x?

For your information, the first 20 significant figures of the cube root in
question are:
   49793385921817447440

Stephen Tucker.
--
>>> 123.456789 ** (1.0 / 3.0)
4.979338592181744
>>> 123456.789 ** (1.0 / 3.0)
49.79338592181744
>>> 123456789. ** (1.0 / 3.0)
497.9338592181743
>>> 123456789000. ** (1.0 / 3.0)
4979.338592181743
>>> 12345678900. ** (1.0 / 3.0)
49793.38592181742
>>> 1234567890. ** (1.0 / 3.0)
497933.8592181741
>>> 123456789. ** (1.0 / 3.0)
4979338.59218174
>>> 123456789000. ** (1.0 / 3.0)
49793385.9218174
>>> 12345678900. ** (1.0 / 3.0)
497933859.2181739
>>> 1234567890. ** (1.0 / 3.0)
4979338592.181739
>>> 123456789. ** (1.0 / 3.0)
49793385921.81738
>>> 123456789000. ** (1.0 / 3.0)
497933859218.1737
>>> 12345678900. ** (1.0 / 3.0)
4979338592181.736
>>> 1234567890. ** (1.0 / 3.0)
49793385921817.36
>>> 123456789. ** (1.0 / 3.0)
497933859218173.56
>>> 123456789000. ** (1.0 / 3.0)
4979338592181735.0
>>> 12345678900. ** (1.0 / 3.0)
4.979338592181734e+16
>>> 1234567890. ** (1.0 / 3.0)
4.979338592181734e+17
>>> 123456789. ** (1.0 /
3.0)
4.979338592181733e+18
>>> 123456789000. ** (1.0 /
3.0)
4.979338592181732e+19
>>> 12345678900. **
(1.0 / 3.0)
4.9793385921817313e+20
--
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kSE4mNp5KxTEp6SKzpQeBukScLYsmEoDfLpSTuc2Zv8Z3pZQhTm0usq-k4eVquxM08u8VSUX1X6id9IICJHA2B4mzw$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Precision Tail-off?

[Weatherby,Gerard]

All languages that use IEEE floating point will indeed have the same 
limitations, but it is not true that Python3 only uses IEEE floating point. 
Using the Decimal class and cribbing a method from StackOverflow, 
https://stackoverflow.com/questions/47191533/how-to-efficiently-calculate-cube-roots-using-decimal-in-python


import decimal
from decimal import Decimal

decimal.getcontext().prec = 1_000_000


def cube_root(A: Decimal):
guess = (A - Decimal(1)) / Decimal(3)
x0 = (Decimal(2) * guess + A / Decimal(guess * guess)) / Decimal(3.0)
while 1:
xn = (Decimal(2) * x0 + A / Decimal(x0 * x0)) / Decimal(3.0)
if xn == x0:
break
x0 = xn
return xn


float_root = 5 ** (1.0 / 3)
float_r3 = float_root * float_root * float_root
print(5 - float_r3)
five = Decimal(5.0)
r = cube_root(five)
decimal_r3 = r * r * r
print(5 - decimal_r3)


8.881784197001252e-16
1E-99

From: Python-list  on 
behalf of Michael Torrie 
Date: Tuesday, February 14, 2023 at 5:52 PM
To: python-list@python.org 
Subject: Re: Precision Tail-off?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2/14/23 00:09, Stephen Tucker wrote:
> I have two questions:
> 1. Is there a straightforward explanation for this or is it a bug?
To you 1/3 may be an exact fraction, and the definition of raising a
number to that power means a cube root which also has an exact answer,
but to the computer, 1/3 is 0.333 repeating in decimal,
which is some other fraction in binary.  And even rational numbers like
0.2, which are precise and exact, are not in binary
(0.01010101010101010101).  0.2 is .0011011011011011011 on and on forever.

IEEE floating point has very well known limitations.  All languages that
use IEEE floating point will be subject to these limitations.  So it's
not a bug in the sense that all languages will exhibit this behavior.

> 2. Is the same behaviour exhibited in Python 3.x?
Yes. And Java, C++, and any other language that uses IEEE floating point.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jjhLqksliV_IjxQAHxXvdnOLB00sJU_hfHNIfK2U1NK-yO2X2kOxJtk6nbqEzXZkyOPBOaMdIlz_sHGkpA$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python: How to use the 'trace' module programmatically?

[Weatherby,Gerard]

Have you tried the filter options?

“These options may be repeated multiple times.
--ignore-module=
Ignore each of the given module names and its submodules (if it is a package). 
The argument can be a list of names separated by a comma.
--ignore-dir=
Ignore all modules and packages in the named directory and subdirectories. The 
argument can be a list of directories separated by 
os.pathsep.”



From: Python-list  on 
behalf of Peter Slížik 
Date: Wednesday, February 15, 2023 at 12:22 PM
To: python-list@python.org 
Subject: Python: How to use the 'trace' module programmatically?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hello,

I'm trying to analyze complex Python code. For some specific reasons, I
decided to use tracing instead of a debugger.

The first thing I tried was:

python -m trace -t /path/to/file.py

The output of this command turned out to be completely useless. The reason
is that there was a thread running in the background, doing some work
every *0.1
s* and this generated thousands of lines of tracing information. The useful
information (a reaction to my interaction with app GUI) scrolled away in a
blink.

For this reason, I decided to limit the scope of tracing. I did the
following.

The original code:

def caller():
print("I'm the caller.")
callee()
def callee():
print("Callee here.")

Code modified for tracing:

import trace

tracer = trace.Tracer(
count=0,
trace=1,
)
def outer():
print("I'm the caller.")
tracer.runfunc(inner)
def inner():
print("Callee here.")

Now I launched the program and the tracer did not generate any output. I
was hoping that this would provide complete tracing information, but only
for the limited scope of inner().

No success with tracer.run() either.

What I was able to do, when I set count=1, I was able to catch the coverage
data with tracer.results() and write them to a file. But the tracing
information was not generated even in this case.

Am I doing anything wrong?

Peter
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!m_WsH0rFVMLw4RMkVvcu-ZoClOMWVTsdB8E99Qy5Sq7ZZF1iBw5_NpLvorEe3_hYvy2kdDwe2obDr1E2ZjFCM3Of$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: LRU cache

[Weatherby,Gerard]

I think this does the trick:

https://gist.github.com/Gerardwx/c60d200b4db8e7864cb3342dd19d41c9


#!/usr/bin/env python3
import collections
import random
from typing import Hashable, Any, Optional, Dict, Tuple


class LruCache:
"""Dictionary like storage of most recently inserted values"""

def __init__(self, size: int = 1000):
""":param size number of cached entries"""
assert isinstance(size, int)
self.size = size
self.insert_counter = 0
   self.oldest = 0
self._data : Dict[Hashable,Tuple[Any,int]]= {} # store values and age 
index
self._lru: Dict[int, Hashable] = {} # age counter dictionary

def insert(self, key: Hashable, value: Any) -> None:
"""Insert into dictionary"""
existing = self._data.get(key, None)
self._data[key] = (value, self.insert_counter)
self._lru[self.insert_counter] = key
if existing is not None:
self._lru.pop(existing[1], None)  # remove old counter value, if it 
exists
self.insert_counter += 1
if (sz := len(self._data)) > self.size:  # is cache full?
assert sz == self.size + 1
while (
key := self._lru.get(self.oldest, None)) is None:  # index may not 
be present, if value was reinserted
self.oldest += 1
del self._data[key]  # remove oldest key / value from dictionary
del self._lru[self.oldest]
self.oldest += 1  # next oldest index
assert len(self._lru) == len(self._data)

def get(self, key: Hashable) -> Optional[Any]:
"""Get value or return None if not in cache"""
if (tpl := self._data.get(key, None)) is not None:
return tpl[0]
return None


if __name__ == "__main__":
CACHE_SIZE = 1000
TEST_SIZE = 1_000_000
cache = LruCache(size=CACHE_SIZE)

all = []
for i in range(TEST_SIZE):
all.append(random.randint(-5000, 5000))

summary = collections.defaultdict(int)
for value in all:
cache.insert(value, value * value)
summary[value] += 1
smallest = TEST_SIZE
largest = -TEST_SIZE
total = 0
for value, count in summary.items():
smallest = min(smallest, count)
largest = max(largest, count)
total += count
avg = total / len(summary)
print(f"{len(summary)} values occurrences range from {smallest} to 
{largest}, average {avg:.1f}")

recent = set()  # recent most recent entries
for i in range(len(all) - 1, -1, -1):  # loop backwards to get the most 
recent entries
value = all[i]
if len(recent) < CACHE_SIZE:
recent.add(value)
if value in recent:
if (r := cache.get(value)) != value * value:
raise ValueError(f"Cache missing recent {value} {r}")
else:
if cache.get(value) != None:
raise ValueError(f"Cache includes old {value}")

From: Python-list  on 
behalf of Dino 
Date: Wednesday, February 15, 2023 at 3:07 PM
To: python-list@python.org 
Subject: Re: LRU cache
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Thank you Mats, Avi and Chris

btw, functools.lru_cache seems rather different from what I need, but
maybe I am missing something. I'll look closer.

On 2/14/2023 7:36 PM, Mats Wichmann wrote:
> On 2/14/23 15:07, Dino wrote:
>>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jb3Gr2BFAPLJ2YuI5rFdJUtalqWcijhxHAfdmCI3afnLFDdcekALxDYAQwpE1L_JlJBBJ-BB3BuLdoSE$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Precision Tail-off?

[Weatherby,Gerard]

IEEE did not define a standard for floating point arithmetics. They designed 
multiple standards, including a decimal float point one.  Although decimal 
floating point (DFP) hardware used to be manufactured, I couldn’t find any 
current manufacturers. There was a company that seemed to be active until a few 
years ago, but they seem to have gone dark: https://twitter.com/SilMinds



From: Python-list  on 
behalf of Thomas Passin 
Date: Friday, February 17, 2023 at 9:02 AM
To: python-list@python.org 
Subject: Re: Precision Tail-off?
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 2/17/2023 5:27 AM, Stephen Tucker wrote:
> Thanks, one and all, for your reponses.
>
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
>
> Consider an integer N consisting of a finitely-long string of digits in
> base 10.

What you are not considering is that the IEEE standard is about trying
to achieve a balance between resource use (memory and registers),
precision, speed of computation, reliability (consistent and stable
results), and compatibility.  So there have to be many tradeoffs.  One
of them is the use of binary representation.  It has never been about
achieving ideal mathematical perfection for some set of special cases.

Want a different set of tradeoffs?  Fine, go for it.  Python has Decimal
and rational libraries among others.  They run more slowly than IEEE,
but maybe that's a good tradeoff for you.  Use a symbolic math library.
Trap special cases of interest to you and calculate them differently.
Roll your own.  Trouble is, you have to know one heck of a lot to roll
your own, and it may take decades of debugging to get it right.  Even
then it won't have hardware assistance like IEEE floating point usually has.

> Consider the infinitely-precise cube root of N (yes I know that it could
> never be computed unless N is the cube of an integer, but this is a
> mathematical argument, not a computational one), also in base 10. Let's
> call it RootN.
>
> Now consider appending three zeroes to the right-hand end of N (let's call
> it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).
>
> The *only *difference between RootN and RootNZZZ is that the decimal point
> in RootNZZZ is one place further to the right than the decimal point in
> RootN.
>
> None of the digits in RootNZZZ's string should be different from the
> corresponding digits in RootN.
>
> I rest my case.
>
> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.
>
> Stephen Tucker.
>
>
> On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson 
> wrote:
>
>> On Tue, 14 Feb 2023 11:17:20 +, Oscar Benjamin wrote:
>>> On Tue, 14 Feb 2023 at 07:12, Stephen Tucker 
>> wrote:
>> [snip]
 I have just produced the following log in IDLE (admittedly, in Python
 2.7.10 and, yes I know that it has been superseded).

 It appears to show a precision tail-off as the supplied float gets
>> bigger.
>> [snip]

 For your information, the first 20 significant figures of the cube root
>> in
 question are:
 49793385921817447440

 Stephen Tucker.
 --
>>> 123.456789 ** (1.0 / 3.0)
 4.979338592181744
>>> 1234567890. ** (1.0 / 3.0)
 49793385921817.36
>>>
>>> You need to be aware that 1.0/3.0 is a float that is not exactly equal
>>> to 1/3 ...
>> [snip]
>>> SymPy again:
>>>
>>> In [37]: a, x = symbols('a, x')
>>>
>>> In [38]: print(series(a**x, x, Rational(1, 3), 2))
>>> a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
>>>
>>> You can see that the leading relative error term from x being not
>>> quite equal to 1/3 is proportional to the log of the base. You should
>>> expect this difference to grow approximately linearly as you keep
>>> adding more zeros in the base.
>>
>> Marvelous.  Thank you.
>>
>>
>> --
>> To email me, substitute nowhere->runbox, invalid->com.
>> --
>> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jqgolDJWMiHsy0l-fRvM6Flcs478R5LIidNh2fAfa3kuPrtqTm0FC6uQmnUuyWLNypQZd3PkzzGyRzZlkbA$
>>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jqgolDJWMiHsy0l-fRvM6Flcs478R5LIidNh2fAfa3kuPrtqTm0FC6uQmnUuyWLNypQZd3PkzzGyRzZlkbA$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python + Vim editor

[Weatherby,Gerard]

Vim 2% of the time, PyCharm (with VI plugin) 98% of the time.

From: Python-list  on 
behalf of Hen Hanna 
Date: Tuesday, February 21, 2023 at 9:38 PM
To: python-list@python.org 
Subject: Python + Vim editor
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

what editor do you (all) use to write Python code?  (i use Vim)





yesterday (?)  i started noticing this with   Bing  search 
e.g.   i type   [synonymrich]   whch gives several synonyms,
and so  (next)  i try  to replace  [rich]  by
 something like Affluent

so i try tohit  [<-]to erase  [rich]
  but i can't... the [text entering]  box  is frozen.

so i give up and go to Google.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jN9E2ZVh45QRBtJ1BgpTjJlghI54aPL2rcB7dlPx1IFw6h8P3vQXgVcNA0rjIEAt0WIg_9oMc7OLfSbcnoU$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Creating logs with Python

[Weatherby,Gerard]

https://docs.python.org/3/howto/logging.html

From: Python-list  on 
behalf of Bibi 
Date: Wednesday, February 22, 2023 at 9:44 AM
To: python-list@python.org 
Subject: Creating logs with Python
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Hello
I want to store and make available as part of my project, logs, for access to 
data. Do you have any proposals?
Kind regards
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ntBEz59Ey9Aq3lepQ3xSKwSYtD_mYgoMO3OPqxMUrzbHNliAV76yHKsVIbEDpznq3hXNHvGxG3RNUPVxj5k$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Line continuation and comments

[Weatherby,Gerard]

That’s a neat tip. End of line comments work, too

x = (3 > 4  #never
 and 7 == 7  # hopefully
 or datetime.datetime.now().day > 15 # sometimes
 )
print(x)

From: Python-list  on 
behalf of Edmondo Giovannozzi 
Date: Wednesday, February 22, 2023 at 9:40 AM
To: python-list@python.org 
Subject: Re: Line continuation and comments
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

Il giorno mercoledì 22 febbraio 2023 alle 09:50:14 UTC+1 Robert Latest ha 
scritto:
> I found myself building a complicated logical condition with many ands and ors
> which I made more manageable by putting the various terms on individual lines
> and breaking them with the "\" line continuation character. In this context it
> would have been nice to be able to add comments to lines terms which of course
> isn't possible because the backslash must be the last character on the line.
>
> Question: If the Python syntax were changed to allow comments after 
> line-ending
> backslashes, would it break any existing code? I can't think of an example.

Well you can if you use parenthesis like in:
x = 5
a = (x > 3 and
# x < 21 or
 x > 100
 )
You don't need the "\" to continue a line in this case

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kck9yP0ubC7L_tbIUoMY-nkZJlkXFAiZdnjPtekuYQXN6F8K2wFMW5lO1xZ6gYv6vDdsSo5jxy1QYU_T-EgHsOJ6x7TvXQ$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Line continuation and comments

[Weatherby,Gerard]

“
NB my PyCharm-settings grumble whenever I create an identifier which is
only used once (and perhaps, soon after it was established). I
understand the (space) optimisation, but prefer to trade that for
'readability'.
“
I haven’t seen that one. What I get is warnings about:

def is_adult( self )->bool:
LEGAL_AGE_US = 21
return LEGAL_AGE

It doesn’t like LEGAL_AGE_US being all caps if declared in a function.

From: Python-list  on 
behalf of dn via Python-list 
Date: Thursday, February 23, 2023 at 5:46 PM
To: python-list@python.org 
Subject: Re: Line continuation and comments
*** Attention: This is an external email. Use caution responding, opening 
attachments or clicking on links. ***

On 22/02/2023 21.49, Robert Latest via Python-list wrote:
> I found myself building a complicated logical condition with many ands and ors
> which I made more manageable by putting the various terms on individual lines
> and breaking them with the "\" line continuation character. In this context it
> would have been nice to be able to add comments to lines terms which of course
> isn't possible because the backslash must be the last character on the line.
>
> Question: If the Python syntax were changed to allow comments after 
> line-ending
> backslashes, would it break any existing code? I can't think of an example.

Alternative to suggestions thus far: break the complex* logical
condition into 'labelled' components, and then combine those (likely
shorter) into the condition:

if person.is_adult and person.is_qualified and person.has_funds ...

which presumes that at some previous time we have, for example:

def is_adult( self )->bool:
 return 21 <= self.age <= 65

(not that I'd like to see those two 'magic-constants' in anyone's code,
but (hopefully) the idea has been conveyed...)


* "simple is better than..."

NB my PyCharm-settings grumble whenever I create an identifier which is
only used once (and perhaps, soon after it was established). I
understand the (space) optimisation, but prefer to trade that for
'readability'.
--
Regards,
=dn
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ghW7FUX8GJF79keLaMyaVewXcKw3jexuxF-QJh8h564QBAIoi2ez20tIl5fg762Rcfnh-XA4sG53CKt2NYgHpTWlyA$
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?

[Weatherby,Gerard]

I only use asserts for things I know to be true. Nothing is harder to debug 
than when something you know to be true turns out to be… not True. Because I’ll 
check everything else instead of the cause of the bug.

In other words, a failing assert means I have a hole in my program logic.

For that use, the default behavior –telling me which line the assert is on, is 
more than sufficient. Depending on the circumstance, I’ll re-run the code with 
a breakpoint or replace the assert with an informative f-string Exception.



From: Python-list  on 
behalf of Peter J. Holzer 
Date: Saturday, February 25, 2023 at 9:22 AM
To: python-list@python.org 
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) 
values are ?
On 2023-02-25 09:10:06 -0500, Thomas Passin wrote:
> On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
> > On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> > > Sometimes you can use a second parameter to assert if you know what kind 
> > > of
> > > error to expect:
[...]
> > > With type errors, assert may actually give you the information needed:
> > >
> > > > > > c = {"a": a, "b": 2}
> > > > > > assert a > c
> > > Traceback (most recent call last):
> > >File "", line 1, in 
> > > TypeError: '>' not supported between instances of 'list' and 'dict'
> >
> > Actually in this case it isn't assert which gives you the information,
> > it's evaluating the expression itself. You get the same error with just
> >  a > c
> > on a line by its own.
>
> In some cases.  For my example with an explanatory string, you wouldn't want
> to write code like that after an ordinary line of code, at least not very
> often.  The assert statement allows it syntactically.

Yes, but if an error in the expression triggers an exception (as in this
case) the explanatory string will never be displayed.

hp

--
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a more efficient threading lock?

[Weatherby,Gerard]

“I'm no expert on locks, but you don't usually want to keep a lock while
some long-running computation goes on.  You want the computation to be
done by a separate thread, put its results somewhere, and then notify
the choreographing thread that the result is ready.”

Maybe. There are so many possible threaded application designs I’d hesitate to 
make a general statement.

The threading.Lock.acquire method has flags for both a non-blocking attempt and 
a timeout, so a valid design could include a long-running computation with a 
main thread or event loop polling the thread. Or the thread could signal a main 
loop some other way.

I’ve written some code that coordinated threads by having a process talk to 
itself using a socket.socketpair. The advantage is that you can bundle multiple 
items (sockets, file handles, a polling timeout) into a select.select call 
which waits without consuming resources (at least on Linux) until
something interesting happens.


-- 
https://mail.python.org/mailman/listinfo/python-list