Re: Should NoneType be iterable?

[Cameron Simpson via Python-list]

On 21Jun2023 03:01, Greg Ewing  wrote:

On 20/06/23 7:36 pm, Barry wrote:

I have some APIs that do return None or a list.
The None says that a list is not available and that the caller is
responsible with dealing in a application-domain specific with
with that situation.


In that case, the caller should probably be checking for
None rather than blindly trying to iterate over the result.


I wasted some time the other evening on an API which returned a string 
or None. My own API, and the pain it caused tells me that that API 
design choice isn't good (it's an automatic attribute based on a tag, 
returning None if the tag isn't there). My experience so far is that it 
_looks_ handy so that you can safely say "foo.bar" all the time, but as 
soon a you do something with the value you're in for a world of 
None-checking.


I'm rethinking that choice right now. Just the other day I removed a 
setting in a related class which provided an automatic default value 
because, again, while handy for careless use it caused hard to debug 
problems because the default would flow out the call chain until it was 
unsuitable, making the cause hard to trace.


And of course I'm very -1 on None acquiring iteration or other features.  
Fail early, fail often!


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Should NoneType be iterable?

[Cameron Simpson via Python-list]

On 21Jun2023 10:09, Chris Angelico  wrote:

On Wed, 21 Jun 2023 at 09:59, Cameron Simpson via Python-list
 wrote:
I wasted some time the other evening on an API which returned a 
string

or None. My own API, and the pain it caused tells me that that API
design choice isn't good (it's an automatic attribute based on a tag,
returning None if the tag isn't there). My experience so far is that it
_looks_ handy so that you can safely say "foo.bar" all the time, but as
soon a you do something with the value you're in for a world of
None-checking.


https://peps.python.org/pep-0505/



Hmm. Thanks. - Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: isinstance()

[Cameron Simpson via Python-list]

On 03Aug2023 10:14, dn  wrote:
Can you please explain why a multi-part second-argument must be a 
tuple and not any other form of collection-type?


The signature is: isinstance(object, classinfo)
leading to "classinfo" of:

1/ a single class/type, eg int
2/ a tuple of same, eg ( int, str, )
3/ a union type, eg int | str (v3.10+)


help(isinstance) (in 3.8) says class_or_tuple.

I would speculate that requiring a tuple has two advantages:
- it may lend itself to promotion to a set for comparison with the MRO 
  of object (IIRC the "foo in (a,b,c)" is optimised this way also), and 
  this in turn may hint at the Hashable requirement
- it is _frugal_ in what we expect there, leaving open a future meaning 
  for something else in that place (for example, isinstance well 
  predates type annotations, and a looser kind of argument might have 
  precluded this new usage)


There's similar language in this try/except documentation:
file:///Users/cameron/var/doc/python/3.8.0/reference/compound_stmts.html#index-10

 For an except clause with an expression, that expression is 
 evaluated, and the clause matches the exception if the resulting 
 object is “compatible” with the exception. An object is compatible 
 with an exception if it is the class or a base class of the 
 exception object or a tuple containing an item compatible with the 
 exception.


To my mind the tuple requirement lends itself to a distinct syntax (the 
brackets) and frugal use of the meaning of what values can occur there.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Where is the error?

[Cameron Simpson via Python-list]

On 06Aug2023 22:41, Peter J. Holzer  wrote:

Mostly, error messages got a lot better in Python 3.10, but this one had
me scratching my head for a few minutes.

Consider this useless and faulty script:

r = {
   "x": (1 + 2 + 3)
   "y": (4 + 5 + 6)
   "z": (7 + 8 + 9)
}


[...]

Python 3.10 and 3.11 report:


 File "/home/hjp/tmp/foo", line 2
   "x": (1 + 2 + 3)
 ^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?


The error message is now a lot better, of course, but the fact that it
points at the expression *before* the error completely threw me. The
underlined expression is clearly not missing a comma, nor is there an
error before that.


Well, it's hard to underline a token which isn't present. But maybe the 
message could be more evocative:


SyntaxError: invalid syntax. Perhaps you forgot a comma after the 
underlined code?


Is this "clairvoyant" behaviour a side-effect of the new parser or was
that a deliberate decision?


I have the vague impression the new parser enabled the improved 
reporting.


Used to use a Pascal compiler once which was uncannily good at 
suggesting where you'd missing a semicolon.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Where is the error?

[Cameron Simpson via Python-list]

On 07Aug2023 08:02, Barry  wrote:

On 7 Aug 2023, at 05:28, Cameron Simpson via Python-list 
 wrote:
Used to use a Pascal compiler once which was uncannily good at 
suggesting where you'd missing a semicolon.


Was that on DEC VMS? It was a goal at DEC for its compilers to do this well.


No, a PDP-11 using V7 UNIX.


They could output the errors in a machine readable format to allow editors to 
auto fix.
I am learning rust and it is very good at suggesting fixes.
There is a command to apply fixes automatically, cargo fix.


Neat.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Imports and dot-notation

[Cameron Simpson via Python-list]

On 09Aug2023 12:30, Oliver Schinagl  wrote:
Looking at a python projects code and repository layout, we see the 
following directory structure.



/project/core
/project/components/module1
...
/project/components/moduleN
/projects/util

(this is far from complete, but enough to help paint a picture.

Some modules import other modules, and I see (at the very least) two 
(kind of three?) different ways of doing so.


`from project.components.module1 import function1, function2 as func, 
CONST1, CONST2 as CONST`


or maybe even (which has as an advantage that it becomes clear which 
namespace something belongs to


`from project.components.module1 import function1, function2 as 
module1_function2, CONST1, CONST2 as MODULE1_CONST2`


but then it really just becomes personal preference, as the number of 
characters saved on typing is almost negative (due to having a more 
complex import).



but also the dot notation being used

`from project.components import module1`

where obviously the functions are invoked as `module1.function1` etc

I hope the above is clear enough from an example.


Simply put, which of the two would be considered cleaner and more pythonic?


This is almost entirely a programmer preference thing.

The Zen, as usual, offers guideance but not dictates.

As you say, the module1.func form is very clear and has its beauty.  
Personally I lean towards the single name flavour, sometimes with a 
rename. FOr example, with os.path I often go:


from os.path import join as joinpath

and likewise for several other names from there, because the bare names 
are very generic (eg "join"). I see to be alone here. Many other people 
use:


import os.path

and use the full os.path.join in their code, which I find jarring and 
visually noisy.


Now for a bit more thought, looking at PEP8, we notes about imports, 
but sadly PEP8 does not state which method is better/preferred. While 
obviously in the end, it's always the maintainers choice in what they 
prefer, so are tabs vs spaces and PEP8 is opinionated about that too 
(in a good way, even though I'm not a fan :p).


PEP8 is for the stdlib source code; that said it is also the basis for 
most Python coding styles in the wild.


For me, the Zen's "readability counts" is the most important guideline; 
and to this end, your "module1.function" is an entirely reasonable 
response, particularly if you've got a lot of imports. But it does make 
for more verbose code, and that works against readability to a degree.


This is why it's all subjective.

Particular workplaces may mandate particular styles, but in your 
personal code? Do what seem best, and experience will cause that to 
evolve over time.


Also, the style can depend on the code you're working on. For a small 
module with few imports the:


from module1 import funcname

can be a win because the code is short, and this short form of funcname 
is both clear and readable. But in a very long piece of code with many 
imports you might go for module1.funcname for clarity, particularly if 
funcname is generic or overlaps with another similar imported name.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doc call `__init__` as a method rather than function?

[Cameron Simpson via Python-list]

On 15Sep2023 10:49, scruel tao  wrote:

```python

class A:

...   def __init__(self):
... pass
...
```

On many books and even the official documents, it seems that many authors prefer to call `__init__` 
as a "method" rather than a "function".
The book PYTHON CRASH COURSE  mentioned that "A function that’s part of a class is a 
method.", however, ` A.__init__` tells that `__init__` is a function...


As mentioned, methods in Python _are_ functions for use with a class.


A.__init__



a = A()
a.__init__


>
I wonder how can I call `__init__` as? Consider the output above.
Maybe both are OK?


As you can see, they're both legal expressions.

The thing about `__init__` is that it's usually called automatically 
which you make a new object. Try putting a `print()` call in your 
`__init__` method, then make a new instance of `A`.


The purpose of `__init__` is to initialise the object's attribute/state 
after the basic, empty-ish, object is made.


Like other dunder methods (methods named with double underscores front 
and back) it is called automatically for you. In a subclass the 
`__init__` method calls the subperclass `__init__` and then does 
whatever additional things might be wanted by the subclass.


Let's look at what you used above:

>>> A.__init__


Here's we've just got a reference to the function you supposlied with 
the class definition for class `A`.


This:

>>> a = A()
>>> a.__init__


Here's you've accessed the name `__init__` via an existing instance of 
`A`, your variable `a`. At this point you haven't called it. So you've 
got a callable thing which is a binding of the function to the object 
`a` i.e. when you call it, the "bound method" knows that t is associated 
with `a` and puts that in as the first argument (usually named `self`).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Where I do ask for a new feature

[Cameron Simpson via Python-list]

On 19Oct2023 20:16, Bongo Ferno  wrote:
A with statement makes clear that the alias is an alias and is local, 
and it automatically clears the variable after the block code is used.


No it doesn't:

>>> with open('/dev/null') as f:
...   print(f)
...
<_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'>
>>> print(f)
<_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'>
--
https://mail.python.org/mailman/listinfo/python-list

Re: return type same as class gives NameError.

[Cameron Simpson via Python-list]

On 22Oct2023 17:50, Antoon Pardon  wrote:

I have the following small module:
=-=-=-=-=-=-=-=-=-=-=-= 8< =-=-=-=-=-=-=-=-=-=-=-=-=
class Pnt (NamedTuple):
   x: float
   y: float

   def __add__(self, other: PNT) -> Pnt:
   return Pnt(self[0] + other[0], self[1] + other[1])


When this function is defined, the class "Pnt" has not yet been defined.  
That happens afterwards.


You want a forward reference, eg:

def __add__(self, other: PNT) -> "Pnt":

A type checker will resolve this after the fact, when it encounters the 
string in the type annotation.


This message:

NameError: name 'Pnt' is not defined. Did you mean: 'PNT'?

is unfortunate, because you have a very similar "PNT" name in scope. But 
it isn't what you want.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Checking if email is valid

[Cameron Simpson via Python-list]

On 01Nov2023 14:08, Grant Edwards  wrote:

On 2023-11-01, Simon Connah via Python-list  wrote:

I'm building a simple project using smtplib and have a
question. I've been doing unit testing but I'm not sure how to check
if an email message is valid.

[...]

Could someone push me in the right direction please? I just want to
find out if a string is a valid email address.


I confess that I punt "syntactically valid" to email.utils.getaddresses:
https://docs.python.org/3/library/email.utils.html#email.utils.getaddresses

"Deliverable"? I'm prepared to just send to it and hope not to get a 
bounce; delivery of email is, after all, asynchronous. (Even if it were 
going direct to the primary MX, I still hand it to the local mail system 
to deal with.)


"A real person or entity"? A lot of systems do the round trip thing: 
here's a special unique and opaue URL,please visit it to confirm receipt 
of this email (implying email is 'real"). You see this a lot when 
signing up for things. And for plenty of things I generate a random 
throw away address at mailinator.com (looking at you, every "catch up" 
free online TV streaming service who still wants me to log in).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Checking if email is valid

[Cameron Simpson via Python-list]

On 02Nov2023 17:04, Chris Angelico  wrote:

On Thu, 2 Nov 2023 at 15:20, AVI GROSS via Python-list
 wrote:
Yes, it would be nice if there was a syntax for sending a test 
message sort

of like an ACK that is not delivered to the recipient but merely results in
some status being sent back such as DELIVERABLE or NO SUCH USER or even
MAILBOX FULL.


Yes, it would! Spammers would be able to use this syntax to figure out
exactly which addresses actually have real people connected to it. It
would save them so much trouble! Brilliant idea.


Hmm. IIRC...

https://datatracker.ietf.org/doc/html/rfc2821#section-4.1.1.6

I think a lot of mail receivers don't honour this one, for exactly the 
reasons above.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Newline (NuBe Question)

[Cameron Simpson via Python-list]

On 15Nov2023 07:25, Grizzy Adams  wrote:

Have this (from an online "classes" tutorial)


Response inline below.


students = []
grades = []
for s in geographyClass:
students.append(geographyStudent(s))
for s in students:
   grades.append(s.school)
   grades.append(s.name)
   grades.append(s.finalGrade())
   if s.finalGrade()>82:
   grades.append("Pass")
   else:
   grades.append("Fail")
print(grades)

--- End Code Snippit  ---

I have extended (from tutorial) it a bit, I would really like to have a newline
at end of each record, I have searched (and tested) but cant get "\n" 
to give a newline, I get "Mydata\n"


It would be useful to:
- see some of the things you've tried
- what their output actually was
- what output you actually want to achieve


Do I need to replace "append" with "print", or is there a way to get the
newline in as I append to list?


I think you're confusing output (print) with your data (the list of 
grades).


Is the code above genuinely what you're running?

I ask because it looks to me that you're:
- appending all the individual grade fields (school, name, ...) to one 
  long grades list containing all the data from all the students

- you're printing that single enormous list in one go at the end

What I'm imagine you want is one line of grade information per student.

Remember that indentation is important in Python. You're grades code 
looks like this:


grades = []
for s in students:
grades.append(s.school)
grades.append(s.name)
grades.append(s.finalGrade())
if s.finalGrade()>82:
grades.append("Pass")
else:
grades.append("Fail")
print(grades)

This:
- makes an empty list
- gathers up all of the student data
- prints the data in one go

I think you may want to do the first and last steps on a per student 
basis, not just once at the start and the end. So you might want to 
rearrange things:


for s in students:
grades = []
grades.append(s.school)
grades.append(s.name)
grades.append(s.finalGrade())
if s.finalGrade()>82:
grades.append("Pass")
else:
grades.append("Fail")
print(grades)

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Type hints - am I doing it right?

[Cameron Simpson via Python-list]

On 13Dec2023 09:19, Frank Millman  wrote:

I am adding type hints to my code base.

[...]

In the other module I have this -

 def config_database(db_params):

To add a type hint, I now have this -

 def config_database(db_params: configparser.SectionProxy):

To get this to work, I have to add 'import configparser' at the top of 
the module.


I have three separate modules, one for each database, with a subclass 
containing the methods and attributes specific to that database. Each 
one has a connect() method which receives db_params as a parameter. 
Now I have to add 'import configparser' at the top of each of these 
modules in order to type hint the method.


This seems verbose. If it is the correct way of doing it I can live 
with it, but I wondered if there was an easier way.


Not really. It's like any other name - it needs importing if you're 
going to use it.


You can make the hint itself more compact:

from configparser import SectionProxy
...
def config_database(db_params: SectionProxy):

Or you could be a bit more agnostic:

from typing import Mapping
...
def config_database(db_params: Mapping):

since I imagine config_database() would accept any kind of mapping 
(dicts, etc etc).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How would you name this dictionary?

[Cameron Simpson via Python-list]

On 21Jan2024 23:39, bagra...@live.com  wrote:

class NameMe(dict):
   def __missing__(self, key):
   return key


I would need to know more about what it might be used for. What larger 
problem led you to writing a `dict` subclass with this particular 
`__missing__` implementation?

--
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to implement the ** operator on a custom object

[Cameron Simpson via Python-list]

On 08Feb2024 12:21, tony.fl...@btinternet.com  wrote:
I know that mappings by default support the ** operator, to unpack the 
mapping into key word arguments.


Has it been considered implementing a dunder method for the ** 
operator so you could unpack an object into a key word argument, and 
the developer could choose which keywords would be generated (or could 
even generate 'virtual' attributes).


Can you show us why you think that would look like in code?

Note that Python already has `a ** b` to raise `a` to the power of `b`, 
and it has a bunder method `__pow__` which you can define.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to implement the ** operator on a custom object

[Cameron Simpson via Python-list]

On 09Feb2024 18:56, Left Right  wrote:

But, more to the point: extending collections.abc.Mapping may or may
not be possible in OP's case.


We don't yet know if that's what the OP had in mind yet, anyway.


Also, if you are doing this through inheritance, this seems really
convoluted: why not just inherit from dict? -- less methods to
implement, less stuff to import etc.


There's a rule of thumb that we _tend_ not to subclass the builtins; it 
certainly has its pitfalls to do with object creation/initialisation.  
That said, I have some classes which subclass dict, int, str and 
namedtuple.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: A question about import

[Cameron Simpson via Python-list]

On 16Feb2024 20:32, MRAB  wrote:

On 2024-02-16 20:07, Gabor Urban via Python-list wrote:

I need something about modules to be clarified.

Suppose I have written a module eg: ModuleA which imports an other
module, let us say the datetime.

If I import ModuleA in a script, will be datetime imported automatically?


Yes. When a module is imported it can import other modules.


But note that `datetime` does not magicly get put in the script's 
namespace.


Module A:

import datetime

Script:

import A

In the code in module A the name datetime is known and can be used.

In the code in the script the name A is known and can be used. Importing 
A does not magicly set the name datetime in the script's namespace - 
imagine the the pollution!


You _can_ access it as A.datetime because it is in the A module's 
namespace. But really if you just wanted datetime for direct use in the 
script you would import it there too:


import datetime
import A

Note that the datetime module is only actually loaded once. The import 
binds the name into your local namespace like any other variable.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Can one output something other than 'nan' for not a number values?

[Cameron Simpson via Python-list]

On 16Feb2024 22:12, Chris Green  wrote:

I'm looking for a simple way to make NaN values output as something
like '-' or even just a space instead of the string 'nan'.  This would
then make it much easier to handle outputting values from sensors when
not all sensors are present.

So, for example, my battery monitoring program outputs:-

   Battery Voltages and Currents
   Leisure Battery - 12.42 volts  -0.52 Amps
   Starter Battery - 12.34 volts  -0.01 Amps

If the starter battery sensor has failed, or is disconnected, I see:-

   Battery Voltages and Currents
   Leisure Battery - 12.42 volts  -0.52 Amps
   Starter Battery -   nan voltsnan Amps


What I would like is for those 'nan' strings to be just a '-' or
something similar.

Obviously I can write conditional code to check for float('nan')
values but is there a neater way with any sort of formatting string or
other sort of cleverness?


The simplest thing is probably just a function writing it how you want 
it:


def float_s(f):
if isnan(f):
return "-"
return str(f)

and then use eg:

print(f'value is {float_s(value)}')

or whatever fits your code.

Cheers,
Cameron Simpson 

--
https://mail.python.org/mailman/listinfo/python-list

Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference

[Cameron Simpson via Python-list]

On 05Mar2024 20:13, Jacob Kruger  wrote:
Now, what almost seems to be occurring, is that while just manipulating 
the contents of a referenced variable is fine in this context, the 
moment I try to reassign it, that's where the issue is occurring .


Because there are no variable definitions in Python, when you write a 
function Python does a static analysis of it to decide which variables 
are local and which are not. If there's an assignment to a variable, it 
is a local variable.  _Regardless_ of whether that assignment has been 
executed, or gets executed at all (eg in an if-statement branch which 
doesn't fire).


You can use `global` or `nonlocal` to change where Python looks for a 
particular name.


In the code below, `f1` has no local variables and `f2` has an `x` and 
`l1` local variable.


x = 1
l1 = [1, 2, 3]

def f1():
print("f1 ...")
l1[1] = 5   # _not_ an assignment to "l1"
print("in f1, x =", x, "l1 =", l1)

def f2():
print("f2 ...")
x = 3
l1 = [6, 7, 9]  # assignment to "l1"
print("in f2, x =", x, "l1 =", l1)

print("outside, x =", x, "l1 =", l1)
f1()
print("outside after f1, x =", x, "l1 =", l1)
f2()
print("outside after f2, x =", x, "l1 =", l1)

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Variable scope inside and outside functions - global statement being overridden by assignation unless preceded by reference

[Cameron Simpson via Python-list]

On 06Mar2024 15:12, Jacob Kruger  wrote:
So, this does not make sense to me in terms of the following snippet 
from the official python docs page:

https://docs.python.org/3/faq/programming.html

"In Python, variables that are only referenced inside a function are 
implicitly global. If a variable is assigned a value anywhere within 
the function’s body, it’s assumed to be a local unless explicitly 
declared as global."


So, I would then assume that if I explicitly include a variable name 
inside the global statement, then even just assigning it a new value 
should update the variable in the global context, outside the 
function?


Yes. Note that the "global" namespace is the module in which the 
function is defined.


x = 1

def f(n):
global x
x += n

This updates the `x` global variable in the module where `f` was 
defined.


If you import `f` and use it in another module it will _still_ update 
`x` in the original module namespace.

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to Add ANSI Color to User Response

[Cameron Simpson via Python-list]

On 10Apr2024 23:41, Alan Gauld  wrote:

Normally, for any kind of fancy terminal work, I'd say use curses.


My problem with curses is that it takes over the whole terminal - you 
need to manage everything from that point on. Great if you want it (eg 
some full-terminal tool like `top`) but complex overkill for small 
interactive (or not interactive) commands which are basicly printing 
lines of text. Which is what many of my scripts are.


That said, you don't _have_ to use curses to run the whole terminal. You 
can use it to just look up the terminal capabilities and use those 
strings. I haven't tried that for colours, but here's some same code 
from my `cs.upd` module using curses to look up various cursor motion 
type things:


... up the top ...
try:
  import curses
except ImportError as curses_e:
  warning("cannot import curses: %s", curses_e)
  curses = None

... later we cache the available motions ...
try:
  # pylint: disable=no-member
  curses.setupterm(fd=backend_fd)
except TypeError:
  pass
else:
  for ti_name in (
  'vi',  # cursor invisible
  'vs',  # cursor visible
  'cuu1',  # cursor up one line
  'dl1',  # delete one line
  'il1',  # insert one line
  'el',  # clear to end of line
  ):
# pylint: disable=no-member
s = curses.tigetstr(ti_name)
if s is not None:
  s = s.decode('ascii')
self._ti_strs[ti_name] = s

... then a method to access the cache ...
def ti_str(self, ti_name):
  ''' Fetch the terminfo capability string named `ti_name`.
  Return the string or `None` if not available.
  '''
  return self._ti_strs.get(ti_name, None)

... and even later, use the method ...
# emit cursor_up
cursor_up = self.ti_str('cuu1')
movetxts.append(cursor_up * (to_slot - from_slot))

Generally, when I add ANSI colours I do it via a little module of my 
own, `cs.ansi_colour`, which you can get from PyPI using `pip`.


The two most useful items in it for someone else are probably 
`colourise` and `colourise_patterns`. Link:

https://github.com/cameron-simpson/css/blob/26504f1df55e1bbdef00c3ff7f0cb00b2babdc01/lib/python/cs/ansi_colour.py#L96

I particularly use it to automatically colour log messages on a 
terminal, example code:

https://github.com/cameron-simpson/css/blob/26504f1df55e1bbdef00c3ff7f0cb00b2babdc01/lib/python/cs/logutils.py#L824
--
https://mail.python.org/mailman/listinfo/python-list

Re: First two bytes of 'stdout' are lost

[Cameron Simpson via Python-list]

On 11Apr2024 14:42, Olivier B.  wrote:

I am trying to use StringIO to capture stdout, in code that looks like this:

import sys
from io import StringIO
old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()
print( "patate")
mystdout.seek(0)
sys.stdout = old_stdout
print(mystdout.read())

Well, it is not exactly like this, since this works properly


Aye, I just tried that. All good.


This code is actually run from C++ using the C Python API.
This worked quite well, so the code was right at some point. But now,
two things changed:
- Now using python 3.11.7 instead of 3.7.12
- Now using only the python limited C API


Maybe you should post the code then: the exact Python code and the exact 
C++ code.



And it seems that now, mystdout.read() always misses the first two
characters that have been written to stdout.

My first ideas was something related to the BOM improperly truncated
at some point, but i am manipulating UTF-8, so the bom would be 3
bytes, not 2.


I didn't think UTF-8 needed a BOM. Somone will doubtless correct me.

However, does the `mystdout.read()` code _know_ you're using UTF-8? I 
have the vague impression that eg some Windows systems default to UTF-16 
of some flavour, possibly _with_ a BOM.


I'm suggesting that you rigorously check that the bytes->text bits know 
what text encoding they're using. If you've left an encoding out 
anywhere, put it in explicitly.



Hopefully someone has a clue on what would have changed in Python for
this to stop working compared to python 3.7?


None at all, alas. My experience with the Python C API is very limited.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Terminal Emulator

[Cameron Simpson via Python-list]

On 14May2024 18:44, Gordinator  wrote:
I wish to write a terminal emulator in Python. I am a fairly competent 
Python user, and I wish to try a new project idea. What references can 
I use when writing my terminal emulator? I wish for it to be a true 
terminal emulator as well, not just a Tk text widget or something like 
that.


Start with the `pty` module.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Flubbed it in the second interation through the string: range error... HOW?

[Cameron Simpson via Python-list]

On 29May2024 01:14, Thomas Passin  wrote:
Also, it's 2024 ... time to start using f-strings (because they are 
more readable than str.format())


By which Thomas means stuff like this:

print(f'if block {name[index]} and index {index}')

Notice the leading "f'". Personally I wouldn't even go that far, just:

print('if block', name[index], 'and index', index)

But there are plenty of places where f-strings are very useful.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: IDLE: clearing the screen

[Cameron Simpson via Python-list]

On 04Jun2024 22:43, Rob Cliffe  wrote:

import os
def cls(): x=os.system("cls")

Now whenever you type
cls()
it will clear the screen and show the prompt at the top of the screen.

(The reason for the "x=" is: os.system returns a result, in this case 
0.  When you evaluate an expression in the IDE, the IDE prints the 
result.  So without the "x=" you get an extra line at the top of the 
screen containing "0".)


Not if it's in a function, because the IDLE prints the result if it 
isn't None, and your function returns None. So:


def cls():
os.system("cls")

should be just fine.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggested python feature: allowing except in context maneger

[Cameron Simpson via Python-list]

On 13Jun2024 19:44, dieter.mau...@online.de  wrote:

Why not use:
```
try:
 with open()...
   ...
except FileNotFoundError:
 ...
```


This is exactly what the OP was expressing dissatisfaction with.

I'm -1 on the idea myself - not every combination of things needs 
additional syntactic support, and doing stuff like merging an `except` 
with a `wtih` is bound to introduce some weird corner case, complicating 
its semantics.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Anonymous email users

[Cameron Simpson via Python-list]

On 14Jun2024 18:00, avi.e.gr...@gmail.com  wrote:

I notice that in some recent discussions, we have users who cannot be
replied to directly as their email addresses are not valid ones, and I
believe on purpose. Examples in the thread I was going to reply to are:

 henha...@devnull.tb

[...]
I know some here suggest that we only reply to the wider community and 
they

have a point. But I think there is a role for having some conversations
offline and especially when they are not likely to be wanted, or even
tolerated, by many in the community.

Using such fake or invalid emails makes it hard to answer the person
directly or perhaps politely ask them for more info on their request or
discuss unrelated common interests. Worse, when I reply, unless I use
reply-all, my mailer sends to them futilely. When I do the reply-all, I have
to edit out their name or get a rejection.


I often reply-all (meaning to the list and to the author). And edit the 
headers (frankly, often just to discard anything @gmail.com which has 
very stupid spam poolicies).  If I miss an @invalid.com or whatever, 
then whoops.


If I want to reply directly (eg for some kind of feedback rather than a 
list type reply) and they've got a bogus address, well then I don't.  
Supposedly my reply would be of benefit for them or I shouldn't be doing 
it, so their loss. But equally, if they don't want personal off-list 
contact, they've expressed their preference. I should respect that.


Plenty of people have reasons to post anonymously, even to a list like 
this one. Just assume they've got their reasons and move on.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Suggested python feature: allowing except in context maneger

[Cameron Simpson via Python-list]

On 14Jun2024 09:07, Yair Eshel  wrote:

Cameron, I'm not really sure I got your point. I've used the "file not
found" exception as an example for a behavior typical on context managers.
This could be a failure to connect to DB, or threads. It also applies to
any kind of possible exception, whether cased by the context manager itself
or the lines inside it. Long story short, this syntax change is as useful
as context managers are


The example exception is not what bothers me. The syntax change is 
nowhere near as useful as `with` and context managers. They provide an 
excellent idiom for resource usage and release.


Your suggestion complicates the `with` statement and brings only a tiny 
indentation reduction over the `with`-inside-`try` idiom. It brings no 
semantic changes or new features.


That is why I'm -1: the benefit is triviailly small to my eye.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best use of "open" context manager

[Cameron Simpson via Python-list]

On 06Jul2024 11:49, Rob Cliffe  wrote:

try:
    f = open(FileName) as f:
    FileLines = f.readlines()
except FileNotFoundError:
    print(f"File {FileName} not found")
    sys.exit()
# I forgot to put "f.close()" here -:)
for ln in File Lines:
        print("I do a lot of processing here")
        # Many lines of code here .


What about this:

try:
    f = open(FileName) as f:
except FileNotFoundError:
    print(f"File {FileName} not found")
    sys.exit()
with f:
... process the lines here ...

Remember, the `open()` call returns a file object _which can be used as 
a context manager_. It is separate from the `with` itself.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Best use of "open" context manager

[Cameron Simpson via Python-list]

On 07Jul2024 22:22, Rob Cliffe  wrote:
Remember, the `open()` call returns a file object _which can be used 
as a context manager_. It is separate from the `with` itself.

Did you test this?
    f = open(FileName) as f:
is not legal syntax.


No. You're right, remove the "as f:".

it's legal, but doesn't work (trying to access the file after "with f" 
raises the same

    ValueError: I/O operation on closed file.


This astounds me. Code snippet to demo this?

Here's a test script which I've just run now:

FileName = 'foo.txt'
try:
  f = open(FileName)
except FileNotFoundError:
  print(f"File {FileName} not found")
  sys.exit()
with f:
  for line in f:
print("line:", line.rstrip())

Here's the foo.txt file:

here are
some lines of text

Here's the run:

% python3 p.py
line: here are
line: some lines of text
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best use of "open" context manager

[Cameron Simpson via Python-list]

On 07Jul2024 22:22, Rob Cliffe  wrote:
it's legal, but doesn't work (trying to access the file after "with f" 
raises the same

    ValueError: I/O operation on closed file.


Just to this: of course. The with closes the file. But my version runs 
the with after the try/except.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Help needed - - running into issues with python and its tools

[Cameron Simpson via Python-list]

On 03Aug2024 16:34, o1bigtenor  wrote:
So please - - - how do I set up a venv so that I can install and run 
python

3.12
(and other needed programs related to 3.12) inside?


Maybe this github comment will help with this:
https://github.com/orgs/micropython/discussions/10255#discussioncomment-671
--
https://mail.python.org/mailman/listinfo/python-list

Re: python3 package import difference?

[Cameron Simpson via Python-list]

On 07Aug2024 08:35, Tobiah  wrote:

When I do the same import with python3, I get:

   Traceback (most recent call last):
 File "/home/toby/me", line 1, in 
   import rcs.dbi
 File "/usr/regos-1.0/lib/python/rcs/dbi/__init__.py", line 1, in 
   from dbi import *
   ModuleNotFoundError: No module named 'dbi'


Python 3 imports are absolute (they start from the top of the package 
tree and you have no `dbi` at the top). You want a relative import i.e.:


from .dbi import *

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: python3 package import difference?

[Cameron Simpson via Python-list]

On 08Aug2024 21:55, Gilmeh Serda  wrote:
I guess in a sense Py2 was smarter figuring out what whent where and 
where

it came from. Or it was a bad hack that has been removed.


No, Python 2 offered less control.
--
https://mail.python.org/mailman/listinfo/python-list

Re: [Tutor] How to stop a specific thread in Python 2.7?

[Cameron Simpson via Python-list]

On 03Oct2024 22:12, Dan Ciprus (dciprus)  wrote:

I'd be interested too :-).


Untested sketch:

def make_thread(target, *a, E=None, **kw):
'''
Make a new Event E and Thread T, pass `[E,*a]` as the target 
positional arguments.

A shared preexisting Event may be supplied.
Return a 2-tuple of `(T,E)`.
'''
if E is None:
  E = Event()
T = Thread(target=target, args=[E, *a], kwargs=kw)
return T, E

Something along those lines.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best Practice Virtual Environment

[Cameron Simpson via Python-list]

On 05Oct2024 22:27, Ulrich Goebel  wrote:
Debian (or even Python3 itself) doesn't allow to pip install required 
packages system wide,


This is gnerally a good thing. You might modify a critical system-used 
package.



But is it right, that I have to do that for every single user?


No. Just make a shared virtualenv, eg in /usr/local or /opt somewhere.
Have the script commence with:

#!/path/to/the/shred/venv/bin/python

and make it readable and executable.

Problem solved.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Printing UTF-8 mail to terminal

[Cameron Simpson via Python-list]

On 01Nov2024 08:11, Loris Bennett  wrote:

Cameron Simpson  writes:

If you're using the Python email module to parse (or construct) the
message as a `Message` object I'd expect that to happen automatically.


I am using
 email.message.EmailMessage


Noted. That seems like the correct approach to me.

And you are right that encoding for the actual mail which is received 
is

automatically sorted out.  If I display the raw email in my client I get
the following:

 Content-Type: text/plain; charset="utf-8"
 Content-Transfer-Encoding: quoted-printable
 ...
 Subject: =?utf-8?q?=C3=9Cbungsbetreff?=
 ...
 Dies ist eine =C3=9Cbung.


Right. Quoted-printable encoding for the transport.


I would interpret that as meaning that the subject and body are encoded
in the same way.


Yes.


The problem just occurs with the unsent string representation printed to
the terminal.


Yes, and I was thinking abut this yesterday. I suspect that 
`print(some_message_object)` is intended to transcribe it for transport.  
For example, one could write to an mbox file and just print() the 
message into it and get correct transport/storage formatting, which 
includes the qp encoding.


Can you should the code (or example code) which leads to the qp output?  
I suspect there's a straight forward way to get the decoded Unicode, but 
I'd need to see how what you've got was obtained.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Printing UTF-8 mail to terminal

[Cameron Simpson via Python-list]

On 01Nov2024 10:10, Loris Bennett  wrote:

as expected.  The non-UTF-8 text occurs when I do

 mail = EmailMessage()
 mail.set_content(body, cte="quoted-printable")
 ...

 if args.verbose:
 print(mail)

which is presumably also correct.

The question is: What conversion is necessary in order to print the
EmailMessage object to the terminal, such that the quoted-printable
parts are turned (back) into UTF-8?


Do you still have access to `body` ? That would be the original message 
text? Otherwise maybe:


print(mail.get_content())

The objective is to obtain the message body Unicode text (i.e. a regular 
Python string with the original text, unencoded). And to print that.

--
https://mail.python.org/mailman/listinfo/python-list

Re: TkInter Scrolled Listbox class?

[Cameron Simpson via Python-list]

On 04Nov2024 16:32, Ulrich Goebel  wrote:
I would like to build a class ScrolledListbox, which can be packed 
somewhere in ttk.Frames. What I did is to build not really a scrolled 
Listbox but a Frame containing a Listbox and a Scrollbar:


That's what I would build too.


class FrameScrolledListbox(ttk.Frame):
   def __init__(self, *args, **kwargs):
   super().__init__(*args, **kwargs)
   # build Listbox and Scrollbar
   self.Listbox = tk.Listbox(self)
   self.Scrollbar = ttk.Scrollbar(self)

[...]

But it would be a bit nicer to get a class like

class ScrolledListbox(tk.Listbox):
   ...

So it would be used that way:

scrolledListbox = ScrolledListbox(main)
scrolledListbox.config(...)


Probably you want to proxy various methods to the enclosed widgets.  
Possibly you want to do that with parameters in `__init__` also.


Example:

class FrameScrolledListbox(ttk.Frame):
def __init__(self, *frame_args, *, height=None, jump=None, **frame_kw):
super().__init__(*frame_args, **frame_kw)
self.Listbox = tk.Listbox(self, height=height)
self.Scrollbar = ttk.Scrollbar(self, jump=jump)


def config(self, *a, **kw):
return self.Listbox.config(*a, **kw)

and so forth for the various listbox methods you want to proxy to the 
listbox itself. You could pass scroll specific methods to the scrollbar 
as well.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Printing UTF-8 mail to terminal

[Cameron Simpson via Python-list]

On 04Nov2024 13:02, Loris Bennett  wrote:

OK, so I can do:

##
if args.verbose:
   for k in mail.keys():
   print(f"{k}: {mail.get(k)}")
   print('')
   print(mail.get_content())
##

prints what I want and is not wildly clunky, but I am a little surprised
that I can't get a string representation of the whole email in one go.


A string representation of the whole message needs to be correctly 
encoded so that its components can be identified mechanically. So it 
needs to be a syntacticly valid RFC5322 message. Thus the encoding.


As an example (slightly contrived) of why this is important, multipart 
messages are delimited with distinct lines, and their content may not 
present such a line (even f it's in the "raw" original data).


So printing a whole message transcribes it in the encoded form so that 
it can be decoded mechanically. And conservativly, this is usually an 
ASCII compatibly encoding so that it can traverse various systems 
undamaged. This means the text requiring UTF8 encoding get further 
encoded as quoted printable to avoid ambiguity about the meaning of 
bytes/octets which have their high bit set.


BTW, doesn't this:

for k in mail.keys():
print(f"{k}: {mail.get(k)}")

print the quoted printable (i.e. not decoded) form of subject lines?

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Two python issues

[Cameron Simpson via Python-list]

On 05Nov2024 15:48, Raymond Boute  wrote:
Python seem to suffer from a few poor design decisions regarding 
strings and lists that affect the elegance of the language.


(a) An error-prone "feature" is returning -1 if a substring is not 
found by "find", since -1 currently refers to the last item.


`find` is a pretty old API interface. It is what it is. It may obtain 
some of its design choices from C style calls where returning -1 for 
failure was a common idiom.



If "find" is unsuccessful, an error message is the only clean option.


This is not true. Often we want to look for something, and act one way 
or another depending on whether it is found. I've got plenty of loops 
and other tests which more or less go "run until this is not found". It 
is not an error, it is just a circumstance to accomodate.


Moreover, using index -1 for the last item is a bad choice: it should 
be len(s) - 1 (no laziness!).
Negative indices should be reserved for elements preceding the element 
with index 0 (currently not implemented, but a must for orthogonal 
design supporting general sequences).


It is _far_ too late to propose such a change.

Plenty of us are quite hapoy with negative indices. We just view them as 
counting backwarss from the end of the string or sequence instead of 
forwards from the beginning.


(b) When using assignment for slices, only lists with the same length 
as the slice should be acceptable, otherwise an error should be 
given.


There are many many circumstances where we replace a subsequence with a 
different subsequence of different length. Outlawing such a thing would 
remove and extremely useful feature.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Printing UTF-8 mail to terminal

[Cameron Simpson via Python-list]

On 31Oct2024 21:53, alan.ga...@yahoo.co.uk  wrote:

On 31/10/2024 20:50, Cameron Simpson via Python-list wrote:

If you're just dealing with this directly, use the `quopri` stdlib
module: https://docs.python.org/3/library/quopri.html


One of the things I love about this list are these little features
that I didn't know existed. Despite having used Python for over 25
years, I've never noticed that module before! :-)


Heh. And James Parrott caused me to discover the 
`subprocess.run(executable_path)` mode of `run()/Popen()`: a string with 
`shell=False` (the default) is an executable name.

--
https://mail.python.org/mailman/listinfo/python-list

Re: FileNotFoundError thrown due to file name in file, rather than file itself

[Cameron Simpson via Python-list]

On 11Nov2024 18:24, dieter.mau...@online.de  wrote:

Loris Bennett wrote at 2024-11-11 15:05 +0100:

I have the following in my program:
   try:
   logging.config.fileConfig(args.config_file)
   config = configparser.ConfigParser()
   config.read(args.config_file)
   if args.verbose:
   print(f"Configuration file: {args.config_file}")
   except FileNotFoundError:
   print(f"Error: configuration file {args.config_file} not found.  
Exiting.")


Do not replace full error information (including a traceback)
with your own reduced error message.
If you omit your "try ... except FileNotFoundError`
(or start the `except` clause with a `raise`), you
will learn where in the code the exception has been raised
and likely as well what was not found (Python is quite good
with such error details).


Actually, file-not-found is pretty well defined - the except action 
itself is fine in that regard.


[...]

2. In terms of generating a helpful error message, how should one
  distinguish between the config file not existing and the log file not
  existing?


Generally you should put a try/except around the smallest possible piece 
of code. So:


config = configparser.ConfigParser()
try:
config.read(args.config_file)
except FileNotFoundError as e:
print(f"Error: configuration file {args.config_file} not found: {e}")

This way you know that the config file was missing.

Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Printing UTF-8 mail to terminal

[Cameron Simpson via Python-list]

On 31Oct2024 16:33, Loris Bennett  wrote:

I have a command-line program which creates an email containing German
umlauts.  On receiving the mail, my mail client displays the subject and
body correctly:

[...]

So far, so good.  However, when I use the --verbose option to print
the mail to the terminal via

 if args.verbose:
 print(mail)

I get:

 Subject: Übungsbetreff

 Sehr geehrter Herr Dr. Bennett,

 Dies ist eine =C3=9Cbung.

What do I need to do to prevent the body from getting mangled?


That looks to me like quoted-printable. This is an encoding for binary 
transport of text to make it robust against not 8-buit clean transports.  
So your Unicode text is encodings as UTF-8, and then that is encoded in 
quoted-printable for transport through the email system.


Your terminal probably accepts UTF-8 - I imagine other German text 
renders corectly?


You need to get the text and undo the quoted-printable encoding.

If you're using the Python email module to parse (or construct) the 
message as a `Message` object I'd expect that to happen automatically.


If you're just dealing with this directly, use the `quopri` stdlib 
module: https://docs.python.org/3/library/quopri.html


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to stop a specific thread in Python 2.7?

[Cameron Simpson via Python-list]

On 25Sep2024 19:24, marc nicole  wrote:

I want to know how to kill a specific running thread (say by its id)

for now I run and kill a thread like the following:
# start thread
thread1 = threading.Thread(target= self.some_func(), args=( ...,), )
thread1.start()
# kill the thread
event_thread1 = threading.Event()
event_thread1.set()

I know that set() will kill all running threads, but if there was thread2
as well and I want to kill only thread1?


No, `set()` doesn't kill a thread at all. It sets the `Event`, and each 
thread must be checking that event regularly, and quitting if it becomes 
set.


You just need a per-thred vent instead of a single Event for all the 
threads.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to stop a specific thread in Python 2.7?

[Cameron Simpson via Python-list]

On 25Sep2024 22:56, marc nicole  wrote:

How to create a per-thread event in Python 2.7?


Every time you make a Thread, make an Event. Pass it to the thread 
worker function and keep it to hand for your use outside the thread.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Bug in 3.12.5

[Cameron Simpson via Python-list]

On 20Sep2024 12:52, Martin Nilsson  wrote:

The attached program doesn’t work in 3.12.5, but in 3.9 it worked.


This mailing list discards attachments.

Please include your code inline in the message text.

Thanks,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python List is Not Dead

[Cameron Simpson via Python-list]

On 25Dec2024 14:52, Abdur-Rahmaan Janhangeer  wrote:
I have been following discussions on Discourse (discuss.python.org) 
these last times.


I think that it definitely lacks some of the joys of the mailing list:


FYI, it has a very good "mailing list" mode. I use it that was >90% of 
the time, and file both posts from Discourse and posts from python-list 
into my "python" mail folder.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python List is Not Dead

[Cameron Simpson via Python-list]

On 29Dec2024 07:16, Kevin M. Wilson  wrote:

Excuse please, my failure. As I have not been following this discussion, why is the 
subject "Python List Is NOT Dead" a subject for discussion? Has the list been 
moving towards closing?


No, the list's still around. But there was a significant migration to 
https://discuss.python.org/ which is a web forum running Discourse some 
time back, so python-list is a lot quieter these days.

--
https://mail.python.org/mailman/listinfo/python-list