[issue46645] Portable python3 shebang for Windows, macOS, and Linux

[Christoph Reiter]

Christoph Reiter  added the comment:

(MSYS2 maintainer here) If you run a Python script in MSYS2 bash, the shebang 
gets interpreted by bash, which looks up in PATH (which by default doesn't 
include the full Windows PATH to avoid conflicts) and if installed leads to the 
MSYS2 included CPython to be invoked, either the Cygwin based one, or the MinGW 
fork, depending on the active MSYS2 environment.

In short, there is no System CPython involved in any of this.

--
nosy: +lazka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42353] Proposal: re.prefixmatch method (alias for re.match)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I am not convinced. What are examples of using re.match() instead of 
re.search()? How common is this type of errors?

There are perhaps many millions of scripts which use re.match(), deprecating 
re.match() at any time in future would be very destructive, and keeping an 
alias indefinitely would only add more confusion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41259] Find adverbs is not correct on the documentation

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46647] `test_functools` unexpected failures when C `_functoolsmodule` is missing

[Nikita Sobolev]

New submission from Nikita Sobolev :

Reproduction steps:

1. Add to `Setup.local`:

```
*disabled*
_functoolsmodule
```

2. `.configure && make -j`. Then, ensure that this module is not available:

```
» ./python.exe -c 'import _functools' 
Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named '_functools'
```

3. Run `test_functools`:

```
==
ERROR: test_bad_cmp (test.test_functools.TestCmpToKeyC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 905, 
in test_bad_cmp
key = self.cmp_to_key(cmp1)
  ^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

==
ERROR: test_cmp_to_key (test.test_functools.TestCmpToKeyC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 869, 
in test_cmp_to_key
key = self.cmp_to_key(cmp1)
  ^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

==
ERROR: test_cmp_to_key_arguments (test.test_functools.TestCmpToKeyC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 885, 
in test_cmp_to_key_arguments
key = self.cmp_to_key(mycmp=cmp1)
  ^^^
TypeError: cmp_to_key() got multiple values for argument 'mycmp'

==
ERROR: test_hash (test.test_functools.TestCmpToKeyC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 941, 
in test_hash
key = self.cmp_to_key(mycmp)
  ^^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

==
ERROR: test_obj_field (test.test_functools.TestCmpToKeyC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 920, 
in test_obj_field
key = self.cmp_to_key(mycmp=cmp1)
  ^^^
TypeError: cmp_to_key() got multiple values for argument 'mycmp'

==
ERROR: test_sort_int (test.test_functools.TestCmpToKeyC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 926, 
in test_sort_int
self.assertEqual(sorted(range(5), key=self.cmp_to_key(mycmp)),
  ^^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

==
ERROR: test_sort_int_str (test.test_functools.TestCmpToKeyC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 934, 
in test_sort_int_str
values = sorted(values, key=self.cmp_to_key(mycmp))
^^
TypeError: cmp_to_key() takes 1 positional argument but 2 were given

==
ERROR: test_pickle (test.test_functools.TestPartialC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 258, 
in test_pickle
f_copy = pickle.loads(pickle.dumps(f, proto))
  ^^
_pickle.PicklingError: Can't pickle : it's not the 
same object as functools.partial

==
ERROR: test_recursive_pickle (test.test_functools.TestPartialC)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_functools.py", line 343, 
in test_recursive_pickle
pickle.dumps(f, proto)
^^
_pickle.PicklingError: Can't pickle : it's not the 
same object as functools.partial

==
ERROR: test_iterator_usage (test.test_functools.TestReduceC)
-

[issue46647] `test_functools` unexpected failures when C `_functoolsmodule` is missing

[Nikita Sobolev]

Change by Nikita Sobolev :


--
keywords: +patch
pull_requests: +29318
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31141

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36019] test_urllib fail in s390x buildbots: http://www.example.com/

[Nikita Sobolev]

Nikita Sobolev  added the comment:

`test.test_urllib2.MiscTests.test_issue16464` started to fail again:

```
 ==
ERROR: test_issue16464 (test.test_urllib2.MiscTests)
--
Traceback (most recent call last):
  File "/Users/runner/work/cpython/cpython/Lib/contextlib.py", line 155, in 
__exit__
self.gen.throw(typ, value, traceback)
^
  File "/Users/runner/work/cpython/cpython/Lib/test/support/socket_helper.py", 
line 245, in transient_internet
yield
^
  File "/Users/runner/work/cpython/cpython/Lib/test/test_urllib2.py", line 
1799, in test_issue16464
opener.open(request, "1".encode("us-ascii"))

  File "/Users/runner/work/cpython/cpython/Lib/urllib/request.py", line 525, in 
open
response = meth(req, response)
   ^^^
  File "/Users/runner/work/cpython/cpython/Lib/urllib/request.py", line 634, in 
http_response
response = self.parent.error(
   ^^
  File "/Users/runner/work/cpython/cpython/Lib/urllib/request.py", line 563, in 
error
return self._call_chain(*args)
   ^^^
  File "/Users/runner/work/cpython/cpython/Lib/urllib/request.py", line 496, in 
_call_chain
result = func(*args)
 ^^^
  File "/Users/runner/work/cpython/cpython/Lib/urllib/request.py", line 643, in 
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
^^
urllib.error.HTTPError: HTTP Error 404: Not Found

--
Ran 1 test in 0.093s
```

Link: 
https://github.com/python/cpython/runs/5077404591?check_suite_focus=true#step:7:705

Today I had like 3 or 4 different CI failures because of it.

--
nosy: +sobolevn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46648] `test.test_urllib2.MiscTests.test_issue16464` started to fail

[Nikita Sobolev]

New submission from Nikita Sobolev :

Today I've noticed that a lot of CI runs fail because of this test.

Output:

```
==
ERROR: test_issue16464 (test.test_urllib2.MiscTests)
--
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
^
  File "/Users/sobolev/Desktop/cpython/Lib/test/support/socket_helper.py", line 
245, in transient_internet
yield
^
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_urllib2.py", line 1799, in 
test_issue16464
opener.open(request, "1".encode("us-ascii"))

  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 525, in open
response = meth(req, response)
   ^^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 634, in 
http_response
response = self.parent.error(
   ^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 563, in 
error
return self._call_chain(*args)
   ^^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 496, in 
_call_chain
result = func(*args)
 ^^^
  File "/Users/sobolev/Desktop/cpython/Lib/urllib/request.py", line 643, in 
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
^^
urllib.error.HTTPError: HTTP Error 404: Not Found

--
Ran 1 test in 0.448s

FAILED (errors=1)
/Users/sobolev/Desktop/cpython/Lib/test/support/__init__.py:705: 
ResourceWarning: unclosed 
  gc.collect()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
test test_urllib2 failed
test_urllib2 failed (1 error)

== Tests result: FAILURE ==
```

I can also reproduce this failure locally with:

```
./python.exe -m test -v test_urllib2 -m test_issue16464 -u network
```

Related https://bugs.python.org/issue36019

--
components: Tests
messages: 412567
nosy: sobolevn
priority: normal
severity: normal
status: open
title: `test.test_urllib2.MiscTests.test_issue16464` started to fail
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36019] test_urllib fail in s390x buildbots: http://www.example.com/

[Nikita Sobolev]

Nikita Sobolev  added the comment:

I can also reproduce it locally with: `./python.exe -m test -v test_urllib2 -m 
test_issue16464 -u network`

I've opened a new issue for it: https://bugs.python.org/issue46648

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46649] Propagate Python thread name to thread state structure

[Gabriele N Tornetta]

New submission from Gabriele N Tornetta :

For tools like Austin (https://github.com/P403n1x87/austin) it is currently 
quite challenging to derive the name of a thread based on the information 
exposed by the PyThreadState structure and one stored in threading._active. I 
would like to propose propagating the thread name from the Thread object to the 
PyThreadState structure so that profiling information from tools like Austin 
could easily be enriched with the names of each sampled thread.

--
components: C API
messages: 412569
nosy: Gabriele Tornetta
priority: normal
severity: normal
status: open
title: Propagate Python thread name to thread state structure
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46649] Propagate Python thread name to thread state structure

[Gabriele N Tornetta]

Change by Gabriele N Tornetta :


--
keywords: +patch
pull_requests: +29319
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31142

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46649] Propagate Python thread name to thread state structure

[STINNER Victor]

STINNER Victor  added the comment:

bpo-15500 looks like a more generic option, debuggers should be able to 
retrieve thread names.

faulthandler dumps "tstate->thread_id", maybe it should also dump 
"tstate->native_thread_id" and/or the thread name.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46649] Propagate Python thread name to thread state structure

[Gabriele N Tornetta]

Gabriele N Tornetta  added the comment:

Thanks for mentioning that issue, it didn't come up in my search for a similar 
one. Indeed, with the proposals in bpo-15500 implemented, and the fact that 
native_thread_id is already in PyThreadState, retrieving the name should be 
easy and might make for a better implementation than what I'm proposing. 
Furthermore, doing this at the OS level would benefit an even larger set of 
tools. What pushed me to open this issue is the fact that this feature has been 
lacking for quite some time (indeed bpo-15500 dates back to 2012!), so I 
thought I'd propose a different approach in case it would get a quicker 
resolution this way.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46647] `test_functools` unexpected failures when C `_functoolsmodule` is missing

[Christian Heimes]

Christian Heimes  added the comment:

_functoolsmodule is a core bootstrap module and defined in 
Modules/Setup.bootstrap. It has no external dependencies. There is no reason 
and no point to disable the module.

We can safely assume that all modules defined in Modules/Setup.bootstrap are 
always available. (Maybe except for pwd, but that is a different story.)

I propose to close the bug and PR as wontfix.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46632] test_ssl: 2 tests fail on cstratak-CentOS9-fips-x86_64

[Christian Heimes]

Change by Christian Heimes :


--
assignee: christian.heimes -> 
type:  -> behavior
versions: +Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46647] `test_functools` unexpected failures when C `_functoolsmodule` is missing

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Cristian, in this case - is there a reason to keep `skipUnless(c_functools)` 
around? 

If we are sure that it is always available - then it should be always tested. 

We either should have:
1. Cleanly defined skips that work (this PR)
2. Unconditional coverage

Or do you think that some middle-ground is possible?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46649] Propagate Python thread name to thread state structure

[Oleg Iarygin]

Oleg Iarygin  added the comment:

> bpo-15500 looks like a more generic option, debuggers should be able to 
> retrieve thread names.

That option is not viable.

1. > Created on 2012-07-30 16:19 by bra, last changed 2022-01-19 00:32 by pitrou

2. > I don't think this should be done by default as it will break people's 
expectations and, perhaps worse, compatibility.

3. > It is indeed the compatibility that is the worse issue.  The problem is 
what people have gotten used to and may have coded their applications to 
expect/deal with.  I agree with you that most people would *not* find it 
surprising to see the name reflected in the OS, but I don't think the 
convenience of that is worth introducing a potential backward incompatibility.

Any chances?

--
nosy: +arhadthedev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44796] Add __parameters__ and __getitem__ in TypeVar and ParamSpec

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +29321
pull_request: https://github.com/python/cpython/pull/31143

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44796] Add __parameters__ and __getitem__ in TypeVar and ParamSpec

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I have created an alternative PR 31143 which adds special method 
__typing_subst__ in TypeVar and ParamSpec instead of __parameters__ and 
__getitem__. It has almost the same effect except making TypeVar and ParamSpec 
and subscriptable. The Python code in typing.py and _collections_abc.py is 
simplified, but the C code is little changed.

But on other hand, the reasons for PR 27511:

1. We already implemented __getitem__ for a Concatenate alias, although it is 
never documented. It is not even necessary, it is a pure implementation detail 
which makes the code for nested variables substitution simpler. Adding 
__parameters__ and __getitem__ in TypeVar and ParamSpec would have the same 
effect on simplification of the code.

2. There is a use case for subscription of TypeVar. For example:

if version < 2:
   RetType = T
elif version < 3:
   RetType = Optional[T]
else:
   RetType = Optional[T] | Literal[Unknown]

def get_int(precision: int) -> RetType[int]: ...
def get_str(encoding: str) -> RetType[str]: ...
def get_any(converter: Callable[[str], T]) -> RetType[T]: ...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46640] Python can now use the C99 NAN constant

[Mark Dickinson]

Mark Dickinson  added the comment:

The big blocker here is that a platform that fully supports C99 might not 
define the "NAN" macro. I don't think we can require that NAN be defined in 
order for Python to build (which is what the PR currently does, if I'm 
understanding it correctly).

Python deliberately doesn't assume IEEE 754 floating-point. By requiring that 
the C "NAN" macro is present to be able to build Python, we'd be effectively 
requiring IEEE 754 by stealth. (No other common floating-point format has NaNs.)

I'd be fully on board with a decision to require IEEE 754 floating-point for 
Python in future, but that decision would at least need a python-dev discussion 
- we shouldn't sneak it in by the back door.

--
nosy: +rhettinger, tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46650] `priority` in `sched.scheduler` is not sufficiently tested

[Nikita Sobolev]

New submission from Nikita Sobolev :

Right now there only a single test to ensure `priority` works correctly in 
`scheduler`: 
https://github.com/python/cpython/blob/fea7290a0ecee09bbce571d4d10f5881b7ea3485/Lib/test/test_sched.py#L90-L97

It looks like it is not enough. Why?

```
for priority in [1, 2, 3, 4, 5]:
  z = scheduler.enterabs(0.01, priority, fun, (priority,))
scheduler.run()
self.assertEqual(l, [1, 2, 3, 4, 5])
```

This test does not actually test different priorities. It only tests that a 
direct one works correctly. But, this might be a pure coincidence that numbers 
match. They are spawned in this particular order.

What if there are equal numbers? Like `[1, 2, 1]`

I propose adding more examples to this test. PR is on its way.

--
components: Tests
messages: 412577
nosy: sobolevn
priority: normal
severity: normal
status: open
title: `priority` in `sched.scheduler` is not sufficiently tested
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46650] `priority` in `sched.scheduler` is not sufficiently tested

[Nikita Sobolev]

Change by Nikita Sobolev :


--
keywords: +patch
pull_requests: +29322
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31144

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46608] Exclude marshalled-frozen data if deep-freezing to save 300 KB space

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset 9d4161a60ca8b470148ffd6c73e3110a0aa6d66f by Kumar Aditya in 
branch 'main':
bpo-46608: Fix argument parsing in freeze_modules.py (GH-31131)
https://github.com/python/cpython/commit/9d4161a60ca8b470148ffd6c73e3110a0aa6d66f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46479] Implement typing.reveal_locals

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

Withdrawn as there's insufficient demand for this function.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46639] Ceil division with math.ceildiv

[Mark Dickinson]

Mark Dickinson  added the comment:

I'm not convinced that this deserves to be a math module function. I agree that 
`-(-x // y)`, while simple to write, isn't necessarily obvious. But it does 
have some advantages, like not needing an import, and being naturally 
duck-typed, so that it automatically does the right thing for floats, or 
`fractions.Fraction` objects, or `numpy.int64` objects, or SymPy integers. (Not 
for `Decimal` instances, but that's another story.) Unless we were to add a 
whole __ceildiv__ mechanism, a math module implementation would necessarily be 
limited to integers. (Or perhaps integers and floats.)

There's also the "thin end of the wedge" argument: if ceildiv, why not also 
ceilrem, ceildivrem, rounddiv, roundmod, etc.

The main issue with the `-(-x // y)` spelling seems to be discoverability: if 
everyone knew that this was the right way to spell ceiling division, then there 
wouldn't be a problem. And I'm not convinced that a math.ceildiv function would 
necessarily solve the discoverability problem, either.

So maybe the solution is to advertise the `-(-x // y)` pattern better in 
documentation, for example at the point where floor division is introduced in 
the library reference?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46640] Python can now use the C99 NAN constant

[STINNER Victor]

STINNER Victor  added the comment:

> The big blocker here is that a platform that fully supports C99 might not 
> define the "NAN" macro. I don't think we can require that NAN be defined in 
> order for Python to build (which is what the PR currently does, if I'm 
> understanding it correctly).

If a platform doesn't implement NaN, it should define the Py_NO_NAN macro:

/* Py_NAN
 * A value that evaluates to a NaN. On IEEE 754 platforms INF*0 or
 * INF/INF works. Define Py_NO_NAN in pyconfig.h if your platform
 * doesn't support NaNs.
 */
#if !defined(Py_NAN) && !defined(Py_NO_NAN)
   // Use C99 "NAN" constant: quiet Not-A-Number (when supported)
#  define Py_NAN NAN
#endif

Or do you mean that a platform can support NaN but don't define the  
NAN macro?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46640] Python can now use the C99 NAN constant

[Mark Dickinson]

Mark Dickinson  added the comment:

> If a platform doesn't implement NaN, it should define the Py_NO_NAN macro

Ah. In that case your PR description (and the PR news entry) is misleading:

> Building Python now requires a C99  header file providing the
> NAN constant.

Please could you update them?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46639] Ceil division with math.ceildiv

[Vladimir Feinberg]

Vladimir Feinberg  added the comment:

Mark, I will say I'm pretty sympathetic to the feature-bloat avoidance
perspective here, and if the outcome here is to improve docs, that's still
a win, I think.

That said, since this thread will become precedent, and I think
`math.ceildiv` is the exactly-appropriate amount of commitment Lib should
make to the function (not __ceildiv__ and not "just" a doc note), let me
try to give `ceildiv` the strongest legs I can think of before we make a
decision.

1. Not needing an import - I don't find importing such a standard library
as `math` that onerous. We're not adding a new package here, just a
function. This skepticism could be applied to any existing library
function. Even `sys.stdout` needs an import.
2. Natural duck typing - I'll admit, this is pretty nice. But if that's the
argument, I'd expect this to work to its fullest extent. Namely, I'd expect
this to "naturally" work for any ring, and it doesn't. Z/nZ is a common one
and np.uint is a more common one where the identity -(-x // y) = ceildiv(x,
y) does not hold. The benefit of `math.ceildiv` is it'd either support it,
or say it doesn't, but at least it's explicit.
3. Thin end of wedge - A priori, I would put ceildiv as special because of
the "resource coverage" use case I described in my initial bug message. A
posteriori, there's a clear "kink" in the graph of usage here: ceildiv
(3033) ,
rounddiv (25)
, roundmod
(7) , ceilrem
(0) ,
ceildivrem
(0) .

But most importantly, let me detail what motivated me to post this. I was
working on unit tests for linear algebra code which blocked its operations.
But to not involve a lot of context, I'll provide a similarly-structured
use case. Say we're making a controller for a game engine GUI and need to
figure out how to paint sprites.

```
# sprite_A.py
class A:
  def get_covering_rectangle():
return self.x, self.y, self.x - (-self.width // GRID_WIDTH), self.y -
(-self.height // GRID_HEIGHT)
```

Especially if I also use `-(-x//y)` elsewhere, this is just asking too much
of the reader. I could leave a comment to the tune of `# Note below is
equivalent to + (-(-x//y)), the ceildiv operator, and this works because x
isn't a uint`. Should I do this at all usage sites? I'd end up factoring
into my own `ceildiv` for clarity, especially if I use this elsewhere, like
a test.

Where should this hand-rolled ceildiv live, if not recreated in everyone's
code? It seems too light to wrap as its own dependency, and we probably
don't want to go down the leftpad
 path. `math`
seems most apt.

On Sat, Feb 5, 2022 at 8:31 AM Mark Dickinson 
wrote:

>
> Mark Dickinson  added the comment:
>
> I'm not convinced that this deserves to be a math module function. I agree
> that `-(-x // y)`, while simple to write, isn't necessarily obvious. But it
> does have some advantages, like not needing an import, and being naturally
> duck-typed, so that it automatically does the right thing for floats, or
> `fractions.Fraction` objects, or `numpy.int64` objects, or SymPy integers.
> (Not for `Decimal` instances, but that's another story.) Unless we were to
> add a whole __ceildiv__ mechanism, a math module implementation would
> necessarily be limited to integers. (Or perhaps integers and floats.)
>
> There's also the "thin end of the wedge" argument: if ceildiv, why not
> also ceilrem, ceildivrem, rounddiv, roundmod, etc.
>
> The main issue with the `-(-x // y)` spelling seems to be discoverability:
> if everyone knew that this was the right way to spell ceiling division,
> then there wouldn't be a problem. And I'm not convinced that a math.ceildiv
> function would necessarily solve the discoverability problem, either.
>
> So maybe the solution is to advertise the `-(-x // y)` pattern better in
> documentation, for example at the point where floor division is introduced
> in the library reference?
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gobot1234]

Gobot1234  added the comment:

I also support this change. I've had to write a lot of code to make 
SpecialForms able to accept my types where the code has to look like:
```py
class Something:
...
def __call__(self, *args, **kwargs):
   raise NotImplementedError
```
I also know this comes up in typing-extensions a fair bit. I think type 
checkers should be enforcing this at type-checking-time not by typing.py 
run-time.

--
nosy: +Gobot1234

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

I agree with removing this check. I suspect it's a holdover from very early 
typing when static types were supposed to be runtime types. Now the check is a 
bug magnet and doesn't serve a useful purpose.

I think we can just remove the tests that check for ints. I don't see a 
principled reason to special-case int literals.

I wonder if we should apply this change to 3.10 and 3.9. It's arguably a 
bugfix, but it's a pretty big change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46651] test_urllib2.test_issue16464() fails randomly

[STINNER Victor]

New submission from STINNER Victor :

test_urllib2.test_issue16464() fails randomly.

It uses http://www.example.com/ server.

Instead, I proposed to use http://httpbin.org/post URL which is written to 
support HTTP POST.

$ ./python -m test test_urllib2 -u all -v -m test_issue16464
==
ERROR: test_issue16464 (test.test_urllib2.MiscTests)
--
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
^
  File "/home/vstinner/python/main/Lib/test/support/socket_helper.py", line 
245, in transient_internet
yield
^
  File "/home/vstinner/python/main/Lib/test/test_urllib2.py", line 1799, in 
test_issue16464
opener.open(request, "1".encode("us-ascii"))

  File "/home/vstinner/python/main/Lib/urllib/request.py", line 525, in open
response = meth(req, response)
   ^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 634, in 
http_response
response = self.parent.error(
   ^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 563, in error
return self._call_chain(*args)
   ^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 496, in 
_call_chain
result = func(*args)
 ^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 643, in 
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
^^
urllib.error.HTTPError: HTTP Error 404: Not Found

--
components: Tests
messages: 412586
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_urllib2.test_issue16464() fails randomly
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46651] test_urllib2.test_issue16464() fails randomly

[STINNER Victor]

STINNER Victor  added the comment:

Oh, bpo-46648 was already opened.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> `test.test_urllib2.MiscTests.test_issue16464` started to fail

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46648] `test.test_urllib2.MiscTests.test_issue16464` started to fail

[STINNER Victor]

STINNER Victor  added the comment:

I close my bpo-46651 as a duplicate of this issue. Copy of my message.

test_urllib2.test_issue16464() fails randomly.

It uses http://www.example.com/ server.

Instead, I proposed to use http://httpbin.org/post URL which is written to 
support HTTP POST.

$ ./python -m test test_urllib2 -u all -v -m test_issue16464
==
ERROR: test_issue16464 (test.test_urllib2.MiscTests)
--
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
^
  File "/home/vstinner/python/main/Lib/test/support/socket_helper.py", line 
245, in transient_internet
yield
^
  File "/home/vstinner/python/main/Lib/test/test_urllib2.py", line 1799, in 
test_issue16464
opener.open(request, "1".encode("us-ascii"))

  File "/home/vstinner/python/main/Lib/urllib/request.py", line 525, in open
response = meth(req, response)
   ^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 634, in 
http_response
response = self.parent.error(
   ^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 563, in error
return self._call_chain(*args)
   ^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 496, in 
_call_chain
result = func(*args)
 ^^^
  File "/home/vstinner/python/main/Lib/urllib/request.py", line 643, in 
http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
^^
urllib.error.HTTPError: HTTP Error 404: Not Found

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46648] `test.test_urllib2.MiscTests.test_issue16464` started to fail

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +29323
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31146

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46642] typing: tested TypeVar instance subclass TypeError is incidental

[Gregory Beauregard]

Gregory Beauregard  added the comment:

The reason this test passed before is a bit confusing. Run the following code 
standalone to see where the type(TypeVar('T'))(name, bases, namespace) check is 
coming from.
```
class TypeVar:
def __init__(self, name, *constraints):
# in actual TypeVar, each constraint is run through
# typing._type_check, casuing TypeError via not callable() 
print(repr(constraints))

class V(TypeVar("T")):
pass
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46642] typing: tested TypeVar instance subclass TypeError is incidental

[Gregory Beauregard]

Change by Gregory Beauregard :


--
keywords: +patch
pull_requests: +29324
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31148

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46639] Ceil division with math.ceildiv

[Tim Peters]

Tim Peters  added the comment:

GMP's mpz has 18 functions of this form. These are the 6 "ceiling" flavors:

c_divmod
c_div
c_mod

c_divmod_2exp
c_div_2exp
c_mod_2exp

The suggestion here is for c_div.

There are 6 more for floor rounding (with prefix "f_" instead of "c_"), and 
another 6 for truncation ("to-zero" rounding, with prefix "t_"). Curiously 
enough, there's no direct support for any form of "round to nearest".

So that's where this ends ;-)

I personally almost never use -(-x // y). Because it's a bit obscure, and in 
real life y is always known to be positive, so the nearly obvious (x + y - 1) 
// y works fine.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46642] typing: tested TypeVar instance subclass TypeError is incidental

[Gregory Beauregard]

Gregory Beauregard  added the comment:

Fixing this test unblocks bpo-46644

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46187] Optionally support rounding for math.isqrt()

[Tim Peters]

Tim Peters  added the comment:

I've been keeping my eyes open. The only mariginally relevant thing I've 
noticed is Hart's "one line factoring" method:

http://wrap.warwick.ac.uk/54707/1/WRAP_Hart_S1446788712000146a.pdf

That wants the _ceiling_ of the square root. And in another place it wants to 
know "is it a perfect square?". But the part that wants the ceiling doesn't 
care whether it's a perfect square, while the part asking "is it a perfect 
square?" doesn't care what the ceiling may be.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46639] Ceil division with math.ceildiv

[Mark Dickinson]

Mark Dickinson  added the comment:

[Tim]
> Because it's a bit obscure, and in real life y is always known to be 
> positive, so the nearly obvious (x + y - 1) // y works fine.

Whereas I find (x + y - 1) // y less obvious at first sight than -(-x // y). 
:-) I don't care about negative y - that's not my reason for preferring -(-x // 
y). The preference comes from the fact that -(-x // y) still does the right 
thing for non-integral cases.

[Vladimir]
> Say we're making a controller for a game engine GUI and need to
figure out how to paint sprites. [...]

For this example, I'd probably just use `ceil(x / y)`. For "real world" things 
with x and y representing counts of something tangible (pixels, work items, row 
or column count of a matrix, lines of text, bytes of memory used, ...), you 
have to go quite a long way before `ceil(x / y)` gives you the wrong answer due 
to floating-point errors. E.g. if you know the quotient is no larger than 
10**6, you're safe for all y <= 10**10. (Or vice versa: if you know the 
quotient is at most 10**10, then you're safe for y <= 10**6.)

> not __ceildiv__ [...]

It would be a little odd (but only a little) to have __floor__, __ceil__, and 
__floordiv__ overloads, but not __ceildiv__. It probably wouldn't be long 
before someone requested it.

I'll quieten down now and wait to see what other people think.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46648] `test.test_urllib2.MiscTests.test_issue16464` started to fail

[Gregory Beauregard]

Change by Gregory Beauregard :


--
nosy: +GBeauregard
nosy_count: 2.0 -> 3.0
pull_requests: +29325
pull_request: https://github.com/python/cpython/pull/31148

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46639] Ceil division with math.ceildiv

[Tim Peters]

Tim Peters  added the comment:

I expect "obviousness" is mostly driven by background here. You know, e.g., 
that ceil(x) = -floor(-x) for any real x, and the application to integer 
division is just a special case of that. I expect programmers mostly don't know 
that, though. And Python having floor integer division is unusual among 
programming languages. Everyone coming from, say, C, has seen the (i + j - 1)/j 
"idiom" over and over, where "the usual" truncating integer division is the 
rule (and they know too that `i` and `j` are positive). Familiarity breeds 
"obviousness" too :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Christian Heimes  added the comment:


New changeset 96b344c2f15cb09251018f57f19643fe20637392 by Christian Heimes in 
branch 'main':
bpo-40280: Address more test failures on Emscripten (GH-31050)
https://github.com/python/cpython/commit/96b344c2f15cb09251018f57f19643fe20637392


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38941] xml.etree.ElementTree.Element inconsistent warning for bool

[Jacob Walls]

Change by Jacob Walls :


--
keywords: +patch
nosy: +jacobtylerwalls
nosy_count: 6.0 -> 7.0
pull_requests: +29326
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/31149

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38941] xml.etree.ElementTree.Element inconsistent warning for bool

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46642] typing: tested TypeVar instance subclass TypeError is incidental

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

Under the same failing int test cases before there were 2 more cases next to 
them that fail:

with self.assertRaises(TypeError):
ClassVar[int, str]
with self.assertRaises(TypeError):
Final[int, str]

These fail because tuple literals are not callable(). There is code that 
clearly intends for this to be the case: 
https://github.com/python/cpython/blob/96b344c2f15cb09251018f57f19643fe20637392/Lib/typing.py#L486

I can either remove support for this runtime check or change the implementation 
of Final et al to reject tuple literals. I will do the latter for now.

For 
https://github.com/python/cpython/blob/bf95ff91f2c1fc5a57190491f9ccdc63458b089e/Lib/test/test_typing.py#L4262-L4263
 I think the best approach is to just remove these tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

Further questions: the msg argument in _type_check now wouldn't be used for 
anything! It was only used in the case where a type wasn't callable(). I think 
it should be removed. I'm also a bit negative on disallowing tuples in the case 
of e.g. Final and such since it complicates implementing tuple types in Python 
down the line if desired.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46652] Use code.co_qualname to provide richer information

[Gabriele N Tornetta]

New submission from Gabriele N Tornetta :

https://bugs.python.org/issue44530 introduced the co_qualname field to code 
objects. This could be used to, e.g. enrich the information provided by 
tracebacks. Consider this simple example

~~~ python
import traceback


class Bogus:
def __init__(self):
traceback.print_stack()
raise RuntimeError("Oh no!")


class Foo:
def __init__(self):
Bogus()


Foo()
~~~

The current output is

~~~
❯ python3.10 test_tb_format.py  
 
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 15, in 
Foo()
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 12, in __init__
Bogus()
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 6, in __init__
traceback.print_stack()
Traceback (most recent call last):
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 15, in 
Foo()
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 12, in __init__
Bogus()
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 7, in __init__
raise RuntimeError("Oh no!")
RuntimeError: Oh no!
~~~

The proposed change is to use the co_qualname field instead of co_name to 
provide more immediate information about the distinct functions __init__, viz.

~~~
❯ ./python test_tb_format.py   
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 15, in 
Foo()
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 12, in 
Foo.__init__
Bogus()
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 6, in 
Bogus.__init__
traceback.print_stack()
Traceback (most recent call last):
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 15, in 
Foo()
^
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 12, in 
Foo.__init__
Bogus()
^^^
  File "/home/gabriele/Projects/cpython/test_tb_format.py", line 7, in 
Bogus.__init__
raise RuntimeError("Oh no!")

RuntimeError: Oh no!
~~~

This makes it clear that two distinct __init__ functions are involved, without 
having to look at sources.

--
components: Interpreter Core
messages: 412598
nosy: Gabriele Tornetta, pablogsal
priority: normal
severity: normal
status: open
title: Use code.co_qualname to provide richer information
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46652] Use code.co_qualname to provide richer information

[Gabriele N Tornetta]

Change by Gabriele N Tornetta :


--
keywords: +patch
pull_requests: +29327
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31150

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Gregory Beauregard  added the comment:

I made a draft pull request where I went ahead and added a check to disallow 
tuple literals. This is basically already disallowed for types by restrictions 
on `__getitem__` because Union[typeform]->type needs to be different from 
Union[type,type]->Union[type,type].

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32442] file_open unc

[Mike Auty]

Change by Mike Auty :


--
title: Result of pathlib.Path.resolve() with UNC path is not very useful -> 
file_open unc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46653] sys.path entries normalization in site.py doesn't follow POSIX symlink behaviour

[Josselin Poiret]

New submission from Josselin Poiret :

Whenever sys.prefix contains a symlink followed by a '..', the corresponding 
part of sys.path entries will not refer to the parent directory of the 
directory pointed to by the symlink, but rather to the directory in which the 
symlink is. Thus, it will be impossible to import standard Python modules 
installed at sys.prefix, among other things.

Here is an example:
Suppose you have installed Python with prefix /usr. Create a symlink 
/tmp/symlink pointing to /usr/lib, and launch `PYTHONHOME=/tmp/symlink/.. 
python3`. In that REPL, `import warnings` will fail to find the correct module, 
as evidenced by the value of `sys.path` containing entries such as 
`/tmp/lib/python3.X/` instead of the expected `/usr/lib/python3.X/`.

This issue is caused by the makepath function (among others) in Lib/site.py 
using os.path.abspath instead of os.path.realpath, which does not follow POSIX 
as the documentation of os.path.normpath (used internally by abspath) suggests.

I propose replacing all four instances of abspath in Lib/site.py to realpath 
instead. This is a breaking change for users that relied on non-conforming 
symlink semantics (although that user-base might be quite small), but in my 
opinion Python should be expected to follow the behaviour of the platform it is 
running on.

This issue was raised while investigating a bug [1] in the relocatable packs 
feature of GNU Guix [2], which makes use of symlinks to achieve relocatability.

[1] https://issues.guix.gnu.org/53258
[2] 
https://guix.gnu.org/en/blog/2018/tarballs-the-ultimate-container-image-format/

--
components: Library (Lib)
messages: 412600
nosy: jpoiret
priority: normal
severity: normal
status: open
title: sys.path entries normalization in site.py doesn't follow POSIX symlink 
behaviour
type: behavior
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32442] Result of pathlib Path.resolve() with UNC path is not very useful

[Mike Auty]

Mike Auty  added the comment:

Sorry for the spam, thought I was in a different text box.  5:(

--
nosy: +ikelos
title: file_open unc -> Result of pathlib Path.resolve() with UNC path is not 
very useful

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46644] typing: remove callable() check from typing._type_check

[Gregory Beauregard]

Change by Gregory Beauregard :


--
keywords: +patch
pull_requests: +29328
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31151

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Mike Auty]

New submission from Mike Auty :

I've found open to have difficulty with a resolved pathlib path:

Example code of:

   import pathlib
   path = "Z:\\test.py"
   with open(path) as fp:
   print("Stock open: works")
   data = fp.read()
   with open(pathlib.Path(path).resolve().as_uri()) as fp:
   print("Pathlib resolve open")
   data = fp.read()

Results in:

Z:\> python test.py
Stock open: works
Traceback (most recent call last):
  File "Z:\test.py", line 12, in 
with open(pathlib.Path(path).resolve().as_uri()) as fp:
FileNotFoundError: [Errno 2] No such file or directory: 
"file://machine/share/test.py"

Interestingly, I've found that open("file:machine/share/test.py") succeeds, 
but this isn't what pathlib's resolve() produces.  It appears as though 
file_open only supports hosts that are local, but will open UNC paths on 
windows with the additional slashes.  This is quite confusing behaviour and 
it's not clear why file://host/share/file won't work, but 
file:host/share/file does.

I imagine this is a long time issue and a decision has already been reached on 
why file_open doesn't support such URIs, but I couldn't find the answer 
anywhere, just issue 32442 which was resolved without clarifying the 
situation...

--
messages: 412602
nosy: ikelos
priority: normal
severity: normal
status: open
title: file_open doesn't handle UNC paths produced by pathlib's resolve() (but 
can handle UNC paths with additional slashes)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Barney Gale]

Barney Gale  added the comment:

Why are you adding `.as_uri()`?

--
nosy: +barneygale

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Eryk Sun]

Eryk Sun  added the comment:

Builtin open() calls C open(). This C function supports whatever path types are 
supported natively. In Windows, C open() calls WinAPI CreateFileW(), which does 
not support "file://" URIs. The Windows API handles it as a relative path, 
which gets resolved against the current working directory. For example:

>>> os.getcwd()
'C:\\Temp'
>>> nt._getfullpathname('file:host/share/file')
'C:\\Temp\\file:\\host\\share\\file'
>>> nt._getfullpathname('file://host/share/file')
'C:\\Temp\\file:\\host\\share\\file'

As to the resolved path somehow working, that generally will not be the case. 
Most filesystems in Windows will reject a path component named "file:" as an 
invalid name. The ":" character is usually disallowed in base file and 
directory names, since some Windows filesystems use it as a delimiter in file 
streams, e.g. "name:stream_name:stream_type". The default data stream in a 
regular file has no stream name, and its stream type is "$DATA". Thus for base 
name "file", the default data stream can be referenced explicitly as 
"file::$DATA". But just "file:", with neither a stream name nor a stream type, 
is an invalid name.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Mike Auty]

Mike Auty  added the comment:

> Why are you adding `.as_uri()`?

The API we provide accepts URIs, so whilst the example seems a little 
contrived, the code itself expects a URI and then calls open (making use of the 
ability to add open handlers).

> Builtin open() calls C open().

As best I can tell the file handler is defined in urllib/request.py as 
file_open.  This appears to do some preprocessing to remove the file scheme and 
(and explicitly throws an exception if there's a host that isn't localhost) 
before it gets to the C open().  I wondered why it didn't check if it was on 
windows and, if so, construct an appropriate path (since quadruple hash I don't 
think adheres to the URI RFC, but seems to open correctly)?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Mike Auty]

Mike Auty  added the comment:

My bad, sorry, I realized I was conflating open with urllib.request.urlopen.  I 
believe the issue still exists though, sorry for the confusion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Mike Auty]

Mike Auty  added the comment:

Here's the revised code sample:

import pathlib
import urllib.request

path = "Z:\\test.py"

print(f"Stock open: {pathlib.Path(path).as_uri()}")
with urllib.request.urlopen(pathlib.Path(path).as_uri()) as fp:
data = fp.read()

print(f"Pathlib resolved open: {pathlib.Path(path).resolve().as_uri()}")
with urllib.request.urlopen(pathlib.Path(path).resolve().as_uri()) as fp:
data = fp.read()

and here's the output:

Z:\> python test.py
Stock open: file:///Z:/test.py
Pathlib resolved open: file://host/share/test.py
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\urllib\request.py", line 1505, in 
open_local_file
stats = os.stat(localfile)
FileNotFoundError: [WinError 2] The system cannot find the file specified: 
'\\share\\test.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "Z:\test.py", line 14, in 
with urllib.request.urlopen(pathlib.Path(path).resolve().as_uri()) as 
fp:
File "C:\Program Files\Python310\lib\urllib\request.py", line 216, in 
urlopen
return opener.open(url, data, timeout)
File "C:\Program Files\Python310\lib\urllib\request.py", line 519, in open
response = self._open(req, data)
File "C:\Program Files\Python310\lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Program Files\Python310\lib\urllib\request.py", line 496, in 
_call_chain
result = func(*args)
File "C:\Program Files\Python310\lib\urllib\request.py", line 1483, in 
file_open
return self.open_local_file(req)
File "C:\Program Files\Python310\lib\urllib\request.py", line 1522, in 
open_local_file
raise URLError(exp)
urllib.error.URLError: 

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Eryk Sun]

Change by Eryk Sun :


--
Removed message: https://bugs.python.org/msg412604

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Barney Gale]

Barney Gale  added the comment:

urllib uses nturl2path under the hood. On my system it seems to return 
reasonable results for both two and four leading slashes:

>>> nturl2path.url2pathname('host/share/test.py')
'host\\share\\test.py'
>>> nturl2path.url2pathname('//host/share/test.py')
'host\\share\\test.py'

(note that urllib strips the `file:` prefix before calling this function).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45948] Unexpected instantiation behavior for xml.etree.ElementTree.XMLParser(target=None)

[Jacob Walls]

Change by Jacob Walls :


--
keywords: +patch
nosy: +jacobtylerwalls
nosy_count: 1.0 -> 2.0
pull_requests: +29329
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31152

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46652] Use code.co_qualname to provide richer information

[Andre Roberge]

Change by Andre Roberge :


--
nosy: +aroberge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] file_open doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Mike Auty]

Mike Auty  added the comment:

I can confirm that url2pathname work with either number of slashes, and that 
open_file appears to have had the file: removed.

However, in even if the check in open_file were bypassed, it calls 
open_local_file, which then strips any host before calling url2pathname, 
meaning the host will never be included if only two slashes are used.

host, file = _splithost(url)
localname = url2pathname(file)

This is what seems to cause the issue when attempting to open 
file://server/host/file.ext on windows, even though 
file:server/host/file.ext open just fine.

The problem that I found, and was in bug #32442, is that pathlib only ever 
returns two slashes, which despite being a valid and correctly formed url, 
can't be opened by urllib.request.urlopen().  Since there doesn't seem to be an 
issue with opening these files (given it works for file:server...) and 
since nt2pathname will produce the correct result, it feels as though open_file 
should have special code on windows to allow servers to be accepted by the file 
handler (open_local_file should probably stay as is to not change the API too 
much).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Mike Auty]

Change by Mike Auty :


--
title: file_open doesn't handle UNC paths produced by pathlib's resolve() (but 
can handle UNC paths with additional slashes) -> urllib.request.urlopen doesn't 
handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with 
additional slashes)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Eryk Sun]

Eryk Sun  added the comment:

In FileHandler.file_open(), req.host is the host name, which is either None or 
an empty string for a local drive path such as, respectively, 
"file:/Z:/test.py" or "file:///Z:/test.py". The value of req.selector never 
starts with "//", for which file_open() checks, but rather a single slash, such 
as "/Z:/test.py" or "/share/test.py". This is a bug in file_open(). Due to this 
bug, it always calls self.open_local_file(req), even if req.host isn't local. 
The distinction shouldn't matter in Windows, which supports UNC paths, but 
POSIX has to open a path on the local machine (possibly a mount point for a 
remote path, but that's irrelevant). In POSIX, if the local machine 
coincidentally has the req.selector path, then the os.stat() and open() calls 
will succeed with a bogus result.

For "file://host/share/test.py", req.selector is "/share/test.py". In Windows, 
url2pathname() converts this to r"\share\test.py", which is relative to the 
drive of the process current working directory. This is a bug in 
open_local_file() on Windows. For it to work correctly, req.host has to be 
joined back with req.selector as the UNC path "//host/share/test.py". Of 
course, this need not be a local file in Windows, so Windows should be exempted 
from the local file limitation in file_open().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46554] Add append keyword argument to Path.write_text() and Path.write_bytes()

[Keelung Yang]

Keelung Yang  added the comment:

OK. Since most are opposed to append kwarg, I close this issue.
Thanks ALL!

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Eryk Sun]

Eryk Sun  added the comment:

> file://server/host/file.ext on windows, even though 
> file:server/host/file.ext open just fine.

For r"\\host\share\test.py", the two slash conversion 
"file://host/share/test.py" is correct according to RFC80889 "E.3.1.  URI 
with Authority" [1]. In this case, req.host is "host", and req.selector is 
"/share/test.py". 

The four slash version "file:host/share/test.py" is a known variant for a 
converted UNC path, as noted in RFC8089 "E.3.2.  URI with UNC Path". In 
this case, req.host is an empty string, and req.selector is 
"//host/share/test.py". There's another variant that uses 5 slashes for a UNC 
path, but urllib (or url2pathname) doesn't support it.

---
[1] https://datatracker.ietf.org/doc/html/rfc8089

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's resolve() (but can handle UNC paths with additional slashes)

[Barney Gale]

Barney Gale  added the comment:

Agree with the previous analysis. Just noting that:

>>> nturl2path.pathname2url('host\\share\\test.py')
'host/share/test.py'

So four slashes are produced by the urllib code, whereas pathlib only produces 
two.

According to wikipedia, both the two- and four-slash variants are in active 
usage. As we can see within Python itself! :P

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46654] urllib.request.urlopen doesn't handle UNC paths produced by pathlib's as_uri() (but can handle UNC paths with additional slashes)

[Barney Gale]

Change by Barney Gale :


--
title: urllib.request.urlopen doesn't handle UNC paths produced by pathlib's 
resolve() (but can handle UNC paths with additional slashes) -> 
urllib.request.urlopen doesn't handle UNC paths produced by pathlib's as_uri() 
(but can handle UNC paths with additional slashes)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40479] Port _hashlib to OpenSSL 3.0.0

[Zackery Spytz]

Change by Zackery Spytz :


--
nosy: +ZackerySpytz
nosy_count: 3.0 -> 4.0
pull_requests: +29330
pull_request: https://github.com/python/cpython/pull/31153

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40479] Port _hashlib to OpenSSL 3.0.0

[Zackery Spytz]

Zackery Spytz  added the comment:

There is a missing call to va_end() in 443b308fee088e21bbf472c376c5c9e3648f916c 
(which is undefined behavior).  PR 31153 fixes this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34191] argparse: Missing subparser error message should be more clear

[Jacob Walls]

Jacob Walls  added the comment:

Duplicate of issue29298 (fixed)

--
nosy: +jacobtylerwalls

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46652] Use code.co_qualname to provide richer information

[Dennis Sweeney]

Dennis Sweeney  added the comment:

Similar changes at bpo-40679 accidentally broke Cython when it was assumed that 
co_qualname was non-null, which was then fixed by defaulting to co_name in that 
case. I don't know if Cython still produces cases like that, but we should make 
sure not to needlessly break such cases if it does.

--
nosy: +Dennis Sweeney

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46648] `test.test_urllib2.MiscTests.test_issue16464` started to fail

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46648] `test.test_urllib2.MiscTests.test_issue16464` started to fail

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests:  -29325

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45447] Make IDLE recognize .pyi stub files (and .pyw) as python source

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I broaden the issue title scope and will correspondingly restrict the PR scope. 
 'Recognition' applies to open and save dialogs, syntax marking, and module 
browsing.  (The latter needs fixing for .pyw also.)

Improving the handling of no extension source with initial #! line will be a 
different issue.

--
title: IDLE: Support syntax highlighting for .pyi stub files -> Make IDLE 
recognize .pyi stub files (and .pyw) as python source

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46655] typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms

[Gregory Beauregard]

New submission from Gregory Beauregard :

typing.TypeAlias is allowed to be bare, but it's not listed in the list of 
types in typing._type_check that are allowed to be bare. This means it's 
possible to reach the wrong error `TypeError: Plain typing.TypeAlias is not 
valid as type argument` at runtime.

Examples offhand:
from typing import TypeAlias, get_type_hints
class A:
a: "TypeAlias" = int
get_type_hints(A)

from typing import Annotated, TypeAlias
b: Annotated[TypeAlias, ""] = int

There's likely more and/or more realistic ways to trigger the problem. Anything 
that triggers typing._type_check on typing.TypeAlias will give this error 
(TypeError: Plain typing.TypeAlias is not valid as type argument).

I will fix this by adding TypeAlias to the list of typing special forms allowed 
to be bare/plain. I intend to move these to their own named var to reduce the 
chance of types not getting added in the future.

--
components: Library (Lib)
messages: 412618
nosy: GBeauregard
priority: normal
severity: normal
status: open
title: typing.TypeAlias is not in the list of allowed plain _SpecialForm 
typeforms
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46655] typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms

[Gregory Beauregard]

Change by Gregory Beauregard :


--
nosy: +Jelle Zijlstra

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46655] typing.TypeAlias is not in the list of allowed plain _SpecialForm typeforms

[Gregory Beauregard]

Change by Gregory Beauregard :


--
nosy: +kj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com