[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Mario]

Mario  added the comment:

Unfortunately the underlying cause of this issue has not been addressed, nor 
discussed.

There is now a way to workaround the different behaviour in Windows and Linux 
and it is possible to use the new call to make virtual environment work in 
Windows as they already do in Linux.

Problem is that application will have to be change to actually implement the 
workaround.

I still think this difference should be addressed directly.

--

___
Python tracker 
<https://bugs.python.org/issue34725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Mario]

New submission from Mario :

According to the doc Py_GetProgramFullPath() should return the full path of the 
program name as set by Py_SetProgramName().

https://docs.python.org/3/c-api/init.html#c.Py_GetProgramFullPath

This works well in Linux, but in Windows it is always the name of the current 
executable (from GetModuleFileNameW).

This is because the 2 files Modules/getpath.c and PC/getpathp.c have completely 
different logic in calculate_program_full_path() vs get_program_full_path().

This difference is harmless when running in the normal interpreter 
(python.exe), but can be quite dramatic when embedding python into a C 
application.

The value returned by Py_GetProgramFullPath() is the same as sys.executable in 
python.

Why this matters? For instance in Linux virtual environments work out of the 
box for embedded applications, while they are completely ignored in Windows.

python -m venv abcd

and then if I run my app inside the (activated) abcd environment in Linux I can 
access the same modules as if I were executing python, while in Windows I still 
get the system module search path.

If you execute the attached program in Linux you get

EXECUTABLE /tmp/abcd/bin/python3
PATH ['/usr/lib/python37.zip', '/usr/lib/python3.7', 
'/usr/lib/python3.7/lib-dynload', '/tmp/abcd/lib/python3.7/site-packages']

in Windows

EXECUTABLE 
c:\TEMP\vsprojects\ConsoleApplication1\x64\Release\ConsoleApplication1.exe
PATH ['C:\\TEMP\\venv\\abcd\\Scripts\\python37.zip', 'C:\\Python37\\Lib', 
'C:\\Python37\\DLLs', 'c:\\TEMP\\vsprojects\\ConsoleApplication1\\x64\\Relea
se', 'C:\\Python37', 'C:\\Python37\\lib\\site-packages']

with a mixture of paths from the venv, system and my app folder.
But more importantly site-packages comes from the system (bad!).

This is because site.py at lines 454 uses the path of the interpreter to locate 
the venv configuration file.

So in the end, virtual environments work out of the box in Linux even for an 
embedded python, but not in Windows.

--
components: Interpreter Core, Windows
files: poc.c
messages: 325666
nosy: mariofutire, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Py_GetProgramFullPath() odd behaviour in Windows
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47814/poc.c

___
Python tracker 
<https://bugs.python.org/issue34725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Mario]

Mario  added the comment:

On 18/09/2018 19:24, Steve Dower wrote:
> 
> Steve Dower  added the comment:
> 
> That executable doesn't appear to be in a virtual environment - you should be 
> running C:\TEMP\venv\abcd\Scripts\python.exe
> 
> Does that resolve your problem?
> 

Nope,

I am *not* running python, I am running a C app which embeds the python 
interpreter.
I am running exactly

c:\TEMP\vsprojects\ConsoleApplication1\x64\Release\ConsoleApplication1.exe

In a later comment you say the behaviour of Py_GetProgramFullPath is 
intentional: which behaviour? 
Windows? Linux? or the fact that they behave differently?

I guess that if there were a way to force Py_GetProgramFullPath() it would 
solve my problem, because 
I could direct site.py towards the correct virtual environment.

If sys.executable becomes None for embedded python (without the ability to set 
it), then virtual 
environments wont work at all, which would be sad.

--

___
Python tracker 
<https://bugs.python.org/issue34725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Mario]

Mario  added the comment:

On 21/09/2018 21:44, Steve Dower wrote:
> 
> Steve Dower  added the comment:
> 
> I meant returning the full name of the process is intentional. But you're 
> right that overriding it should actually override it.
> 
> I found the prior bug at issue33180, but I'm closing it in favour of this 
> one. I don't have fully fleshed out semantics in my mind for all the cases to 
> handle here, but I hope that we soon reach a point of drastically simplifying 
> getpath and can align the platforms better at that point.
> 
> Meanwhile I'll leave this open in case anyone wants to work on a targeted fix.
> 

So you are saying that the Windows behaviour (+ ability to overwrite) is 
intentional.
This looks to me in contrast to what the doc says under 
https://docs.python.org/3/c-api/init.html#c.Py_GetProgramFullPath.

Moreover I am not sure what Py_SetProgramName() is meant to do then.

The problem in my opinion is that we are trying to fit 2 things in the same 
field: the real 
executable name and the root of the python installation (which could be a 
virtual environment as well).
In python.exe the 2 are the same (or linked), but for embedded applications 
they are not.

Remember that site.py uses the sys.executable as "root of the python 
installation" to derive the 
path and handle virtual environments.

I think that if these 2 concepts were separated, it would be much easier to 
explain the desired 
behaviour and find a valid implementation in Window and Linux.

Let's say sys.executable is the full name of the process and sys.python_root is 
the folder from 
which to derive all the paths.

It is probably too big of a change, but it might be useful to write down the 
ideal behaviour before 
thinking of a pragmatic solution.

Andrea

--

___
Python tracker 
<https://bugs.python.org/issue34725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Mario]

Mario  added the comment:

Is there any agreement on what is wrong with the current code.

The key in my opinion is the double purpose of sys.executable and that in Linux 
and Windows people have taken the two different points of view, so they are 
both right and wrong at the same time.

--

___
Python tracker 
<https://bugs.python.org/issue34725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Mario]

Mario  added the comment:

On 08/10/2018 17:54, Steve Dower wrote:
> 
> Steve Dower  added the comment:
> 
>> Py_SetProgramName() should be a relative or absolute path that can be used 
>> to set sys.executable and other values appropriately.
> 
> Key point here is *can be*, but it doesn't have to be. Given it has fallbacks 
> all the way to "python"/"python3", we can't realistically use it as 
> sys.executable just because it has a value.
> 
> And right now, it's used to locate the current executable (which is 
> unnecessary on Windows), which is then assumed to be correct for 
> sys.executable. Most embedding cases require *this* assumption to be 
> overridden, not the previous assumption.

I still would like my use case to be acknowledged.

site.py uses the value of sys.executable to set up a virtual environment, which 
is a very valuable 
thing even in an embedded cases.

This constraint is strong enough to force it to point to python.exe or python3 
as it would normally 
do in a scripted (non embedded case).

I still believe the 2 concepts should be decoupled to avoid them clashing and 
having supporters of 
one disagreeing with supporters of the other.

Andrea

--

___
Python tracker 
<https://bugs.python.org/issue34725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Mario]

Mario  added the comment:

On 10/10/2018 01:11, Steve Dower wrote:
> 
> Steve Dower  added the comment:
> 
> We'll need to bring in venv specialists to check whether using it outside of 
> Py_Main() is valid. Or perhaps you could explain what you are actually trying 
> to do?
> 

Sure

1) Create a virtual environment ("python -m venv")
2) Activate
2) Pip install some modules
3) Try to use them form inside an embedded application (e.g. the one I attached)
4) Do it in Linux and Windows

Result

Works in Linux, fails in Windows.

Reason in site.py

https://github.com/python/cpython/blob/73870bfeb9cf350d84ee88bd25430c104b3c6191/Lib/site.py#L462

sys.executable is used to construct the correct search path.

Looking at the sys.path from inside an embedded application is very instructive 
and you can see in 
the first post why the failure in windows.

Andrea

--

___
Python tracker 
<https://bugs.python.org/issue34725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34725] Py_GetProgramFullPath() odd behaviour in Windows

[Mario]

Mario  added the comment:

On 13/10/2018 17:37, Steve Dower wrote:
> 
> Steve Dower  added the comment:
> 
> I meant why are you using an embedded application with a virtual environment? 
> What sort of application do you have that requires users to configure a 
> virtual environment, rather than providing its own set of libraries?
> 
> The embedding scenarios I'm aware of almost always want privacy/isolation 
> from whatever a user has installed/configured, so that they can work reliably 
> even when users modify other parts of their own system. I'm trying to 
> understand what scenario (other than "I am an interactive Python shell") 
> would want to automatically pick up the configuration rather than having its 
> own configuration files/settings.

Does it really matter who owns main(), whether it is in python.exe or in some 
other C app.

This is exactly how you described, users want to use some C application which 
will call into python 
using some (user defined) python modules to execute some tasks which are 
scriptable.
And they want to be able to do in a confined environment where they can install 
the exact set of 
packages they require. And it is possible at the same time to set up multiple 
environments where 
different versions are tested independently.

There is as well the totally independent scenario where the app ships exactly 
what it needs, but 
there are some ways in between where one can script an app and in doing so you 
might need packages 
that the app itself knew nothing about.

For another example have a look at JEP
https://github.com/ninia/jep/search?q=virtual&unscoped_q=virtual

This is a way to call python from Java: same problem above, people might want 
to run it in a virtual 
environment and the only way to do this now is to manually set up PYTHONHOME, 
but it is pretty weak 
and does not replicate exactly what happens with virtual environments (e.g. 
inherit system's 
site-packages).

Again, in Linux, JEP works out of the box with no need to tell it about virtual 
environments, 
Py_Initialise() finds it (if they are indeed present) with absolutely no extra 
configuration (no 
need to change PYTHONPATH).

Andrea

--

___
Python tracker 
<https://bugs.python.org/issue34725>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Mario Corchero]

Mario Corchero  added the comment:

Great! I'll put something together then. If you have any preference about the 
implementation or any pointer on the way you think should be done, please let 
me know.

--

___
Python tracker 
<https://bugs.python.org/issue41906>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +28943
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30756

___
Python tracker 
<https://bugs.python.org/issue41906>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11875] OrderedDict.__reduce__ not threadsafe

[Mario Juric]

New submission from Mario Juric :

The implementation of OrderedDict.__reduce__() in Python 2.7.1 is not thread 
safe because of the following four lines:

tmp = self.__map, self.__root
del self.__map, self.__root
inst_dict = vars(self).copy()
self.__map, self.__root = tmp

If one thread is pickling an OrderedDict, while another accesses it, a race 
condition occurs if the accessing thread accesses the dict after self.__map and 
self.__root have been delated, and before they've been set again (above).

This leads to an extremely difficult bug to diagnose when using 
multiprocessing.Queue to exchange OrderedDicts between multiple processes 
(because Queue uses a separate feeder thread to do the pickling).

The fix seems relatively easy -- use:

inst_dict = vars(self).copy()
del inst_dict['_OrderedDict__map'], inst_dict['_OrderedDict__root']

instead of the above four lines.

PS: This issue+fix may also apply to Python 3.x, although I haven't tested it 
there.

--
components: Library (Lib)
messages: 134023
nosy: mjuric
priority: normal
severity: normal
status: open
title: OrderedDict.__reduce__ not threadsafe
type: behavior
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue11875>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11875] OrderedDict.__reduce__ not threadsafe

[Mario Juric]

Mario Juric  added the comment:

Hi Raymond,

Excellent! Many thanks for such a quick fix, this has been bugging us for 
months!

--

___
Python tracker 
<http://bugs.python.org/issue11875>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3451] Asymptotically faster divmod and str(long)

[Pernici Mario]

Pernici Mario <[EMAIL PROTECTED]> added the comment:

I have translated in C the algorithm for long division by
Burnikel and Ziegler (BZ), using the Python version fast_div.py
and the suggestions by Mark.

Here is a benchmark for divmod(p. q), p = 7**np, q = 5**nq
digits = q_digits = p_digits/2; the time taken by Python division
is normalized to 1
tests on Debian, BZ with Python-2.6b3
Pentium 1.60GHzAthlon XP 2600+ Athlon 64 3800+
digits  BZ  fast_div   BZ  fast_divBZ  fast_div
500 1.011.27   1.001.181.001.21
700 0.881.22   0.761.080.811.14
10000.821.17   0.721.040.761.10
20000.660.85   0.550.730.590.78
40000.510.62   0.430.520.450.56
1   0.320.38   0.310.370.330.39
2   0.240.25   0.230.250.250.27
10  0.140.14   0.110.110.120.12

BZ starts being faster than Python division around 2000 bits, apart
from two cases:
 for q with less than 4000 bits and p much smaller than q**2
 x_divrem is faster than divmod_pos;
 for p very much larger than q**2 x_divrem becomes
 again faster.
I put a bound in long_divrem to fall back to x_divrem in those cases,
based on tests on my laptop Pentium 1.60GHz.

The treatment of exceptions is very rudimentary;
I use a simple macro STUB_EXC to return NULL,
but without releasing memory.
Help for doing this properly is welcome.

Please find attached the diff to the longobject.c file in Python-2.6b3 .

Mario

--
nosy: +pernici
Added file: http://bugs.python.org/file11423/diff

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3451>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3944] faster long multiplication

[Pernici Mario]

New submission from Pernici Mario <[EMAIL PROTECTED]>:

In this patch x_mul(a, b) uses fewer bit operations for a != b,
asymptotically half of them.
On the three computers I tried the speed-up is around 5% for size=4
and it increases up to 45-60% just below the Karatsuba cutoff,
then it decreases a bit after this cutoff (on one computer the speed-up
is only 16% after KARATSUBA_CUTOFF=70, but raising the cutoff to 140,
for which with the current code the multiplication is also faster,
the speed-up is 45%).

--
files: longobject_diff
messages: 73624
nosy: pernici
severity: normal
status: open
title: faster long multiplication
type: performance
versions: Python 3.0
Added file: http://bugs.python.org/file11567/longobject_diff

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3944>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3944] faster long multiplication

[Pernici Mario]

Pernici Mario <[EMAIL PROTECTED]> added the comment:

Yes, I think that the speed-up is due to reducing the number of 
shifts and masks.

Changing PyLong_SHIFT to 16 would be complicated; for instance in 
v_iadd() carry could not be a digit of 16 bits anymore; writing code
specific for 64 bit machines would surely improve performance;
maybe with PyLong_SHIFT=30 few changes to the code would be needed?

I did not modify the case a = b.

I changed the documentation, which was wrong,
adding detailed bounds on carry
in the various steps to check that it does not overflow.
I corrected the wrong assertion (carry <= PyLong_MASK).

--
keywords: +patch
Added file: http://bugs.python.org/file11595/longobject1.diff

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3944>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3944] faster long multiplication

[Pernici Mario]

Pernici Mario <[EMAIL PROTECTED]> added the comment:

Mark, following your suggestions about using bigger integer types,
I added code to convert Python numbers to arrays of twodigits,
when a 64 bit integer type is supported, and for numbers with size
larger than 20; otherwise the code of the previous patch is used.
This 64 bit integer is used only inside multiplication, so no
modifications need to be made in other parts of the Python code.
Now with numbers with 300 decimal digits or more the speedup is
2x on 32 bit machine, 3x on 64 bit machine (50% and 2x respectively
for squaring).

There is a macro HAVE_INT64 to control if there is a 64 bit type;
the preprocessor instructions should be OK with gcc, but other
compilers might have a 64 bit type and not long long, so HAVE_INT64
is wrongly not defined and one falls back to multiplying arrays of
16 bit digits; these preprocessor instructions need to be fixed.

The speed difference for small integers is small;
here is a summary of some benchmarks on
Pentium M 1.6GHz, Athlon XP 2600+,
Athlon 64 X2 Dual Core 3800+, all with Debian;
speedup of this patch with respect to the current revision
(+ means the patch is faster):
In pybench, SimpleIntegerArithmetic: from -0.5% to +0.5%
SimpleLongArithmetic:  : from -1% to +7%
pystone: from +0.5% to +1.5%

Added file: http://bugs.python.org/file11653/longobject2.diff

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3944>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3451] Asymptotically faster divmod and str(long)

[Pernici Mario]

Pernici Mario <[EMAIL PROTECTED]> added the comment:

In this patch I added to the patch by Mark in issue 3944 the code
in the previous patch, modified to release memory in case of exceptions.

Benchmark for division on Athlon 64 3800+  with respect to Python3.0:
(1)  Python with patch 30bit.patch
(2)  Python with this patch
up to 1000 digits (1) and (2) perform in the same way
digits(1)(2)
2000  4x 5x
4000  4x 7x
1 4x 10x
2 4x 13x
104x 27x

--
keywords: +patch
Added file: http://bugs.python.org/file11773/longobject2.diff

___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3451>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1225769] Proposal to implement comment rows in csv module

[Mario Fasold]

Mario Fasold  added the comment:

Comment lines are a *very* common case in scientific and statistical data. +1 
for the change.

--
nosy: +Mario.Fasold

___
Python tracker 
<http://bugs.python.org/issue1225769>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17013] Allow waiting on a mock

[Mario Corchero]

Mario Corchero  added the comment:

Spoke offline with @xtreak, I'll be sending a PR for this to take over the 
existing one.

@lisroach proposed a new name, EventMock to differentiate it from any awaitable 
async notation.

@michael.foord suggested using `mock_timeout` as the argument.

Discussed also to change the naming of the method to `await_until_any_call` to 
make the semantics similar to the one that `Mock` provides. As 
`await_until_called_with` might mislead the user that it applies only to the 
last call.

--

___
Python tracker 
<https://bugs.python.org/issue17013>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17013] Allow waiting on a mock

[Mario Corchero]

Change by Mario Corchero :


--
pull_requests: +15715
pull_request: https://github.com/python/cpython/pull/16094

___
Python tracker 
<https://bugs.python.org/issue17013>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38346] Wrong behavior when using `assert_called_with` with mutable object

[Mario Corchero]

Mario Corchero  added the comment:

This might be painful in certain scenarios, like when using wraps on functions 
that modify the arguments:

```
def func(d):
  return d.pop("key")
>>> def func(d):
...   return d.pop("key")
...
>>> m = Mock(wraps=func)
>>> m({"key": 1})
1
>>> m.assert_called_with({"key": 1})
#raises
```

But I think "not fixing" this through copy is reasonable, especially when doing 
copy can also break assertions on objects that cannot be copied, which can 
happen if they implement their own __copy__ and some other situations. 
Additionally, copy does not fully capture "the value of the object when it was 
passed" for custom types.

A copying mock was published under pypi in 
https://github.com/wimglenn/copyingmock but doesn't seem to get a lot of 
attention, if this was interesting by users it could be added as a new type of 
Mock, or maybe just a mixin that users could add to any existing mock if they 
wished.

--

___
Python tracker 
<https://bugs.python.org/issue38346>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38346] Wrong behavior when using `assert_called_with` with mutable object

[Mario Corchero]

Mario Corchero  added the comment:

Thanks! We can look at adding a copying mock if we see people needing it, but 
yeah, adding copy by default would be quite complex if we don't want to break 
existing users.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38346>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Mario Corchero]

Mario Corchero  added the comment:

Vinay would you consider a patch for logging where dictConfig allows taking 
objects directly in addition to the reference id?

That would allow the following:

```

def noErrorLogs(param):
return 1 if param.levelno < 40 else 0

logconfig_dict = {
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"stream": "ext://sys.stdout",
"filters": [noErrorLogs]
}
},
"root": {"level": "DEBUG", "handlers": ["console"]},
"version": 1,
}
dictConfig(logconfig_dict)
```

or alternatively passing them on declaration:

```
logconfig_dict = {
'filters': {
'myfilter': noErrorLogs,
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"stream": "ext://sys.stdout",
"filters": ["myfilter"]
}
},
"root": {"level": "DEBUG", "handlers": ["console"]},
"version": 1,
}
dictConfig(logconfig_dict)
```

I'm happy to put a patch together if that looks good to you.

--
nosy: +mariocj89

___
Python tracker 
<https://bugs.python.org/issue41906>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38757] mocking an exception, arguments do not seem to be passed to the mock

[Mario Corchero]

Mario Corchero  added the comment:

The reason why it seems that "no arguments are beeing passed" is because the 
exception is not being raised by you, but by mock itself when the exception is 
trying to be created. When an exception type is passed as side_effect, the mock 
modules raises such exception on the callable (the creation of the initial 
exception) To confirm this, just try removing the "raise" and leave the 
creation of the exception only.

I'd suggest that you use `wraps` rather than `side_effect`. That will make your 
example work.

Alternatively, if you need to use `side_effect`, you can use a wrapper so mock 
won't raise an exception when it sees that one:

```

def wrapper(*args, **kwargs):
return MockError(*args, **kwargs)

patcher = patch('__main__.ProductionError', side_effect=wrapper)

```


I think this can be closed as a non-issue.

--

___
Python tracker 
<https://bugs.python.org/issue38757>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21600] mock.patch.stopall doesn't work with patch.dict

[Mario Corchero]

Mario Corchero  added the comment:

Makes total sense, I think we should get this for 3.9.
Not sure I'll backport this, even if a bugfix it might cause unexpected changes 
(Though I'd be OK with it, given that calling stop twice causes no issue).

I'm happy to push a PR with the proposed change and some tests if you want 
@xtreak. This is quite a simple fix we can get through in a short time.

--

___
Python tracker 
<https://bugs.python.org/issue21600>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21600] mock.patch.stopall doesn't work with patch.dict

[Mario Corchero]

Change by Mario Corchero :


--
pull_requests: +17075
stage: test needed -> patch review
pull_request: https://github.com/python/cpython/pull/17606

___
Python tracker 
<https://bugs.python.org/issue21600>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39222] unittest.mock.Mock.parent is broken or undocumented

[Mario Corchero]

Mario Corchero  added the comment:

@cjw296 or @michael.foord might know why the attribute was exposed as plain 
"parent". It was added more than 8 years ago and I am unnable to follow the git 
commit history.

They should then decide whether this is intenteded to be used by the users 
(which I never have used on the init TBH) and therefore document it or whether 
it is a private name on the init and if in that case it would be worth trying 
to mangle it.

In the meantime, xtreak suggested workflow is probably the best to go.

--

___
Python tracker 
<https://bugs.python.org/issue39222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39222] unittest.mock.Mock.parent is broken or undocumented

[Mario Corchero]

Mario Corchero  added the comment:

If you refer to https://bugs.python.org/issue35357, this issue refers to 
`parent` at the time of creation of the mock. In the init it is still 
referenced as "parent".

The question is whether we want to also mangle "parent" in the __init__, as it 
seems it is not expected to be part of the API.

--

___
Python tracker 
<https://bugs.python.org/issue39222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39222] unittest.mock.Mock.parent is broken or undocumented

[Mario Corchero]

Mario Corchero  added the comment:

> My suggestion would be to add `parent` to the docs @xtreak links to as a way 
> to resolve this issue.

+1, we should probably add it on the docs of the constructor here 
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock as it 
is done for name and having a section like the on for name.

--

___
Python tracker 
<https://bugs.python.org/issue39222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39578] MagicMock specialisation instance can no longer be passed to new MagicMock instance

[Mario Corchero]

Mario Corchero  added the comment:

Having not looked deeply at it but with the reproducer, running a quick bisect, 
it points to this commit: 
https://github.com/python/cpython/commit/77b3b7701a34ecf6316469e05b79bb91de2addfa
 
I'll try having a look later at the day if I can manage to free some time.

--

___
Python tracker 
<https://bugs.python.org/issue39578>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17013] Allow waiting on a mock

[Mario Corchero]

Mario Corchero  added the comment:

For the record, I have no strong preference over either implementation. 
@voidspace preferred offline the new mock class, but sadly the rationale is 
lost in the physical word.

--

___
Python tracker 
<https://bugs.python.org/issue17013>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42654] Add folder for python customizations: __sitecustomize__

[Mario Corchero]

New submission from Mario Corchero :

Following the conversations in https://bugs.python.org/issue33944, wanted to 
discuss the support for a __sitecustomize__ folder that will hold python 
scripts to be executed at startup. Similar to `sitecustomize.py` but allowing 
different stakeholders of a Python installation to add themselves.

This is basically a "supported way" of the current abuse of pth files to add 
startup code.

How will this be useful?
- Support administrators to add multiple "sitecustomize.py" files. As an 
example, today we are basically appending to sitecustomize when we need some 
additional behaviour.
- Support for library owners that need some startup customization like 
betterexceptions.
- Tools that include an interpreter like virtualenv or things like PyOxidizer 
by allowing them to customize the interpreter they expose to users.

It basically offers a better alternative to the currently abused feature of 
code execution in pth.

I this is something that is wanted in CPython, from the thread in 
https://bugs.python.org/issue33944 I see some open questions though:

- Look for `__sitecustomize__` only in site paths or in PYTHONPATH?
I'm honestly fine either way, but sightly incline more to @jaraco proposal to 
make this basically be a namespace package walking all its instances.

- Should we have a custom way to disable this? Or are we happy with just  `-S`. 
I think the `-S` is fine, it offers a similar behaviour to sitecustomize.

If you want to see "how it feels", see 
https://github.com/mariocj89/cpython/tree/pu/__sitecustomize__ (It's not 
finished).

If it seems interesting I would love to put a PR through. With this, we might 
be able to eventually remove code execution in pth files!

--
components: Library (Lib)
messages: 383129
nosy: jaraco, mariocj89, ncoghlan
priority: normal
severity: normal
status: open
title: Add folder for python customizations: __sitecustomize__
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42654] Add folder for python customizations: __sitecustomize__

[Mario Corchero]

Change by Mario Corchero :


--
type:  -> enhancement

___
Python tracker 
<https://bugs.python.org/issue42654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42654] Add folder for python customizations: __sitecustomize__

[Mario Corchero]

Mario Corchero  added the comment:

> Shouldn't this be discussed on Python-Ideas? I'm pretty sure this is a big 
> enough change that it will need a PEP.

Indeed, I wanted to see if there was interest in the feature. That is probably 
a better place to start.

> If you need code run on startup, can't you just put it in the PYTHONSTARTUP 
> file?

My understanding is that PYTHONSTARTUP is only for interactive executions. 
Additionally, it suffers from the same issue as sitecustomize. It is a single 
file that won't be a real substitute for code execution in pth files.

--

___
Python tracker 
<https://bugs.python.org/issue42654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42654] Add folder for python customizations: __sitecustomize__

[Mario Corchero]

Mario Corchero  added the comment:

Thread open in Python ideas: 
https://discuss.python.org/t/add-folder-for-python-customizations-sitecustomize/6190/5

--

___
Python tracker 
<https://bugs.python.org/issue42654>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42556] unittest.mock.patch() cannot properly mock methods

[Mario Corchero]

Mario Corchero  added the comment:

Right, I believe this is indeed broken.

This code:
```
class A:
def m(self, a=None): pass

from unittest import mock
with mock.patch.object(A, "m", autospec=True):
A.m.side_effect = print
A().m(1)
```

prints: <__main__.A object at 0x7f69cc7dc6a0> 1

Whilst the same code without autospec:
```
class A:
def m(self, a=None): pass

from unittest import mock
with mock.patch.object(A, "m"):
A.m.side_effect = print
A().m(1)
```

prints: 1

This is a rather tricky issue though, as changing the behaviour to pass self 
would break too many people (even if it would probably the be "right" fix).

You can indeed use autospec or just write a descriptor that does the proper 
bound method logic:

```
class A:
def m(self, a=None): pass

def descriptor(self, obj, objtype=None):
self._self = obj
def _(*args):
return self(obj, *args)
return _

from unittest import mock
with mock.patch.object(A, "m"):
A.m.side_effect = print
A.m.__get__ = descriptor
A().m(1)
```

When patching a method at the class level, today Mock is basically hiding 
"self" when used without a spec. We should decide whether:
- that is OK and we want to "fix" autospec to do the same (remove self)
- Leave things as they are, even if not perfect
- Pass self, breaking most assertions across the world (sounds like a bad idea 
initially)
- Create some opt-in parameter or a custom thing like PropertyMock to pass self 
(MethodMock?).

TBH, I don't have a good answer :/. I'm subscribing other more insightful 
people to the issue, maybe this is the way it is intended to work :).

--
nosy: +cjw296, mariocj89, michael.foord, xtreak

___
Python tracker 
<https://bugs.python.org/issue42556>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue896330] pyconfig.h is not placed in --includedir

[Mario Gonzalez]

Mario Gonzalez  added the comment:

hey u, send mi a mail, u need my help
and i need u 
mario the last one
mario.cro...@gmail.com

--
nosy: +Mario Gonzalez

___
Python tracker 
<https://bugs.python.org/issue896330>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41954] [mock] Recursion on mocking inspect.isfunction

[Mario Corchero]

Mario Corchero  added the comment:

Oh, well spotted. We could add some guards around the code in mock to still 
work when key stdlib functionality is mocked out, but this might make the mock 
code quite messy. Specially as we will have to make sure that not only the 
function we call are safe to call (which is easy by just doing import from), 
but that all dependencies of those function are also safe to call.

I'd instead suggest for users to patch in the module where they are using the 
function instead.

I think that once any part of the stdlib is patched by mock, there are no 
guarantees that anything will work unless the patch provides a functioning 
implmentation.

--

___
Python tracker 
<https://bugs.python.org/issue41954>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41958] importlib has not util module

[Mario Idival]

New submission from Mario Idival :

After new version 3.9, the util module from importlib does not exists anymore

>>> import importlib
>>> importlib.util.find_spec('enum')
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: module 'importlib' has no attribute 'util'
>>>

--
components: Library (Lib)
messages: 378121
nosy: marioidival
priority: normal
severity: normal
status: open
title: importlib has not util module
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue41958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41958] importlib has not util module

[Mario Idival]

Change by Mario Idival :


--
type:  -> crash

___
Python tracker 
<https://bugs.python.org/issue41958>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42251] Add threading.gettrace and threading.getprofile

[Mario Corchero]

New submission from Mario Corchero :

When working in a C extension from a non-python created thread which calls 
PyGILState_Ensure as it later calls a Python callback on user code, I wished 
there was a way to set the trace function similar to what a native Python 
thread would do.

We could just call sys.gettrace within the thread, but the application might 
have chosen to set threading.settrace differently.

Same applies for threading.setprofile.

--
components: Library (Lib)
messages: 380273
nosy: mariocj89
priority: normal
severity: normal
status: open
title: Add threading.gettrace and threading.getprofile
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42251>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42251] Add threading.gettrace and threading.getprofile

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +22041
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23125

___
Python tracker 
<https://bugs.python.org/issue42251>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42308] Add threading.__excepthook__ similar to sys.__excepthook__

[Mario Corchero]

New submission from Mario Corchero :

The sys module contains __excepthook__ to recover sys.excepthook if necessary. 
The same is not present in the threading module, even if threading.excepthook 
is exposed.

--
components: Library (Lib)
messages: 380651
nosy: mariocj89, vstinner
priority: normal
severity: normal
status: open
title: Add threading.__excepthook__ similar to sys.__excepthook__
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue42308>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42308] Add threading.__excepthook__ similar to sys.__excepthook__

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +22116
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23218

___
Python tracker 
<https://bugs.python.org/issue42308>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42308] Add threading.__excepthook__ similar to sys.__excepthook__

[Mario Corchero]

Mario Corchero  added the comment:

> I found one interesting usage of sys.__excepthook__ in 
> code.InteractiveInterpreter:

That is exactly what I wished this was there for hehe. We are installing a 
custom version of excepthook and wanted to check if the user had changed it.

--

___
Python tracker 
<https://bugs.python.org/issue42308>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14529] distutils's build_msi command ignores the data_files argument

[Mario Vilas]

New submission from Mario Vilas :

When creating an MSI installer on Python 2.7 (Windows, both in 32 and 64 bits) 
the data_files argument of the setup function is completely ignored. This 
results in broken installations.

--
assignee: eric.araujo
components: Distutils
messages: 157799
nosy: Mario.Vilas, eric.araujo, tarek
priority: normal
severity: normal
status: open
title: distutils's build_msi command ignores the data_files argument
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue14529>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14530] distutils's build_wininst command fails to correctly interpret the data_files argument

[Mario Vilas]

New submission from Mario Vilas :

I tried the following:

setup(

data_files = [(sys.prefix_exec, os.path.join('Win32', 'BeaEngine.dll'))]

  # (... rest of the setup call here...)

)

This works perfectly when running the "python setup.py install". But when 
generating an installer (not MSI but the exe file), the installer places the 
'BeaEngine.dll' in a subdirectory called 'python27'. For 64 bit builds, the 
subdirectory is called 'Python27-x64' instead.

The paths to my python installations are "C:\Python27" and "C:\Python27-x64" 
respectively. The target folders should have been those, not 
"C:\Python27-x64\Python27-x64" which is clearly wrong.

So far my workaround was this:

data_files = [(os.path.join(sys.prefix_exec,'..'), os.path.join('Win32', 
'BeaEngine.dll'))]

But of course, now my setup.py script only works for generating the installer, 
not for installing the module from sources.

--
assignee: eric.araujo
components: Distutils
messages: 157801
nosy: Mario.Vilas, eric.araujo, tarek
priority: normal
severity: normal
status: open
title: distutils's build_wininst command fails to correctly interpret the 
data_files argument
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue14530>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29143] Logger should ignore propagate property for disabled handlers.

[Mario Corchero]

Mario Corchero  added the comment:

Note that "handlers" cannot be disabled. This applies only to loggers.

Also, the following code shows that disabling the logger does indeed prevents 
all logs in emitted by that logger from appearing:

```
import logging

parent_handler = logging.StreamHandler()
child_handler = logging.StreamHandler()

parent_logger = logging.getLogger('parent')
child_logger = logging.getLogger('parent.child')

parent_logger.addHandler(parent_handler)
child_logger.addHandler(child_handler)

child_logger.disabled = True

child_logger.error("wops")
```

Trying to guess what happened, it might be that there was a child of the 
disabled logged and you saw the log trace expecting it was the child, example:

```
import logging

parent_handler = logging.StreamHandler()
child_handler = logging.StreamHandler()

parent_logger = logging.getLogger('parent')
child_logger = logging.getLogger('parent.child')
grandchild_logger = logging.getLogger('parent.child.grandchild')

parent_logger.addHandler(parent_handler)
child_logger.addHandler(child_handler)

child_logger.disabled = True

grandchild_logger.error("wops")
```

Note that disable does not affect how handlers across loggers are called. It 
only disables the logger and therefore prevents from emitting logs that are 
emitted directly through the disabled logger.

Might that be the case? Are you OK with closing this issue if so?

--
nosy: +mariocj89

___
Python tracker 
<https://bugs.python.org/issue29143>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29143] Logger should ignore propagate property for disabled handlers.

[Mario Corchero]

Mario Corchero  added the comment:

Closing as unable to reproduce. Please comment if you can reproduce the issue 
and we can have a further look.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue29143>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37104] logging.Logger.disabled is not documented

[Mario Corchero]

New submission from Mario Corchero :

Just realised the "disabled" attribute of the Logger class is not documented in 
https://docs.python.org/3/library/logging.html#logging.Logger.

Any reason to not have it documented? I'll send a PR otherwise.

This comes as I was trying to point to the docs of the attribute in 
https://bugs.python.org/issue29143

--
assignee: docs@python
components: Documentation
messages: 344007
nosy: docs@python, mariocj89
priority: low
severity: normal
status: open
title: logging.Logger.disabled is not documented
type: enhancement
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue37104>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37104] logging.Logger.disabled is not documented

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +13575
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/13687

___
Python tracker 
<https://bugs.python.org/issue37104>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37104] logging.Logger.disabled is not documented

[Mario Corchero]

Mario Corchero  added the comment:

Thanks! I was wondering about it but saw no comment about it, issue or anything 
in the history. At least we have now this issue :).

Would you like that I move this to `_disabled` and have a setter for `disabled` 
that emits a deprecation warning so we can remove this someday or at least make 
it more explicit that it should not be used?

--

___
Python tracker 
<https://bugs.python.org/issue37104>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37104] logging.Logger.disabled is not documented

[Mario Corchero]

Mario Corchero  added the comment:

Note that even if not in the standard library I've seen this attributed used 
and documented quite often.

Example: https://docs.python-guide.org/writing/logging/#or-print

I would really suggest making it private as mentioned before to make sure this 
does not happen, but if you still prefer just a comment (as in source comment, 
not even documentation) I'll go with that.

--

___
Python tracker 
<https://bugs.python.org/issue37104>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37117] Simplify customization of the logging time through datefmt

[Mario Corchero]

New submission from Mario Corchero :

Users that want to provide a custom template for the timestamp part of logging 
cannot customize the milliseconds part of asctime.

They can use the msecs attribute of the Logger but that effectively requires to 
break the formatting in two parts. Note that it is not necessary to provide 
msecs on the default template as there is code that handles it in 
https://github.com/python/cpython/blob/c8d5bf6c3fa09b43f6a5ee779d493d251dbcc53c/Lib/logging/__init__.py#L603

Something we can do to improve this situation is changing the default converter 
to produce a datetime rather than a timetuple, as strftime of datetime and time 
uses the same template format.

This will allow changing the format including milliseconds through the datefmt 
argument.

formatter = logging.Formatter("%(asctime)s %(message)s", datefmt="%Y%m%d 
%H:%M:%S.%f")

Compare this to the current soltution:

formatter = logging.Formatter("%(asctime)s%(msecs)d %(message)s", 
datefmt="%Y%m%d %H:%M:%S")

Note how you need to split the formatting of time into two different parts. 
This becomes even more diserse if you also want to add something after the time 
formatting (Example: a Z to note UTC).

One more reason why this is quite powerful is that once we get timezones added 
to the standard library we will be able to (in a backward compatible way) just 
pass the timezone when we do `datetime.fromtimestamp`, which will add the 
timezone information to the datetime and users will be able to use %z to add 
the offset of the logging timestamp (Hurray! timestamps with offsets).

Sample implementation: 
https://github.com/mariocj89/cpython/commit/5047d730c0a0dcd6276f40c5b66762071dfcb448

If it looks interesting I can update the docs, add some further tests and send 
the PR. Wanted to confirm it is before putting time into that.

I cannot come with any "downside" of not doing it and I think it will simplify 
considerably the way users can format timestamps in logs. With this change we 
will just be able to say "To change the formatting of timestamps in a 
formatter, just use the datefmt argument".

Thoughts?

--
components: Library (Lib)
messages: 344102
nosy: mariocj89, p-ganssle, vinay.sajip
priority: normal
severity: normal
status: open
title: Simplify customization of the logging time through datefmt
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37117>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37117] Simplify customization of the logging time through datefmt

[Mario Corchero]

Mario Corchero  added the comment:

AFAIK, the converter attribute is only to "be set" with a function of such 
signature. It is not documented that it can be used and it is only used on the 
format time function.

The only situation where this might break someone is if they inherit from 
formatter, substitute the formatTime function by their own implementation and 
rely on the function returning a time tuple. Doing that is relying on the 
implementation of the formatter though, as the docs explain is just something 
to be set.

This could be released as part of a new Python version, I'd be surprised if 
this is an issue and it will improve quite nicely the way users can configure 
the formatting of timestamps.

Even if adding it to the cookbook would "work" I think it would be much better 
to have this change and provide a nicer experience with the default class.

This is your call though! But I'd say it would be quite a nice improvement, was 
speaking with some other devs at PyCon and they seemed quite excited.

--

___
Python tracker 
<https://bugs.python.org/issue37117>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37117] Simplify customization of the logging time through datefmt

[Mario Corchero]

Mario Corchero  added the comment:

> What do you mean by "it is not documented"? It is documented here:

As it says:

"This function uses a user-configurable function to convert the creation time 
to a tuple. By default, time.localtime() is used; to change this for a 
particular formatter instance, set the converter attribute to a function with 
the same signature as time.localtime() or time.gmtime(). To change it for all 
formatters, for example if you want all logging times to be shown in GMT, set 
the converter attribute in the Formatter class." I assumed it means users can 
set such attribute but not something in the formatter class where people will 
do formatter.converter(xyz). If the intention was for users to also be able to 
use the converter attribute then I misunderstood that.

> Obviously, it is also read *somewhere* - otherwise why would you set it?

The implementation of formatTime uses it only at the moment.

> See also questions on Stack Overflow relating to it: 
> https://stackoverflow.com/search?q=%5Bpython%5D+%5Blogging%5D+converter

Through all the questions and answers in stackoverflow they are all setting the 
converter (which will still work) and some (2 I think) only are reading it when 
they are also setting it in the class (Example: 
https://stackoverflow.com/questions/6290739/python-logging-use-milliseconds-in-time-format/6290946#6290946,
 which by chance is effectively trying to solve the same problem brought up 
here).

> But the converter is documented as returning a time tuple,

It just says "set the converter attribute to a function with the same signature 
as time.localtime() or time.gmtime()" there is no reference to what the default 
value has, it could even be None when user is reading it. Of course, you wrote 
it hehe, so if you say the intent is for people to use the default formatter 
that is a different thing :).

>  but that it might break *someone's* code is IMO enough reason not to proceed 
> with it.

I looked hard and could not find any of those in Github (it does not mean that 
there are none indeed). I would agree on not releasing this as a patch release, 
but as part of a new minor I would argue that it is fair. From my point of 
view, it only breaks people depending on the implementation of it and using the 
formatter in a not common way (at least from what was seen in stackoverflow and 
github, there is also no cookbook that recommends doing anything like that).

> It's still a one-liner both ways in your example

Yeah but it requires to deeply understand how Formatter is formatting time 
rather than just using a template.

> Users can always override formatTime in their Formatter subclass to do 
> exactly what they want - it could be a one-time thing that they can then use 
> wherever they want

They need to copy paste this code around on all their scripts/apps or create 
their own library to include this.

> I don't want to give the impression that I'm against *any* change - I'm not

Don't worry, I can totally understand :), if you still think it is not worth 
pursuing this feel free to close the issue. It was an idea I thought it was 
worth pushing but it is totally your call (I won't take it at all personal ^^). 
Thanks for taking the time to answer!

--

___
Python tracker 
<https://bugs.python.org/issue37117>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37251] Mocking a MagicMock with a function spec results in an AsyncMock

[Mario Corchero]

Change by Mario Corchero :


--
nosy: +mariocj89

___
Python tracker 
<https://bugs.python.org/issue37251>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30541] Add restricted mocks to the python unittest mocking framework

[Mario Corchero]

Mario Corchero added the comment:

Whilst I agree that using spec can be used for a similar purpose and I did not 
know about being able to do nested definitions via the arguments (the 
**{"method1.return_value": 1}, really cool!) I find the idea of allowing users 
to halt the mock generation really useful. It is much less disruptive and feels 
more natural.

Compare:
>>> inner_m = Mock(spec=["method2"], **{"method2.return_value": 1})
>>> m = Mock(spec=["method1"], **{"method1.return_value": inner_m})

with: 
>>> m = mock.Mock()
>>> m.method1().method2() = 1
>>> mock.seal(m)


In brief, seal allows users to just add the method to their existing workflow 
where they use generic mocks. Moreover, it is extremely user friendly, many of 
the developers that struggle with the mocking module found seal really helpful.

--

___
Python tracker 
<http://bugs.python.org/issue30541>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35330] When using mock to wrap an existing object, side_effect requires return_value

[Mario Corchero]

Mario Corchero  added the comment:

I can indeed reproduce the issue. The problem seems to be here: 
https://github.com/python/cpython/blob/54ba556c6c7d8fd5504dc142c2e773890c55a774/Lib/unittest/mock.py#L1041

The simplified current logic in that code is:
- call side_effect, save the return value in ret.
- if return_value is not set, call the wraps object and return it.
- return the value in ret from the first step.

That explains why you see your "expected behavior" to happen only when return 
value is set.

Basically, the logic disables the current wrapped object ONLY if return value 
is set.
I am not sure why it was not done for `side_effect` as well.
Trying to perform that change in the sourcecode (nor run wraps if side_effect 
is set) results in no failure from the tests, which might mean it is a bug 

One might claim the code is there because `side_effect` might still be used to 
cause a side effect but not return, I would disagree, especially as setting 
`return_value` totally breaks it. As once the `return_value` is set, both 
`return_value` and `wraps` are ignored and `side_effect` takes preference due 
to line 1044.
Especially as the docs say about side_effect: 
`unless it returns DEFAULT, the return value of this function is used as the 
return value.`

I'd suggest a patch adding the `and not effect` to line 1041, so `side_effect` 
takes preference over `wraps`, the same way `return_value` does today.

The behavior of `side_effect` taken precedence over `return_value` is fine, 
that is how Mock works.



This bug can be reproduced without a class at all, see:

```
from unittest import mock
def func():
print('Original func called')
return "ORIGINAL"

m = mock.Mock(wraps=func)

def side_effect():
print('Side effect func called')
return "SIDE EFFECT"


m.side_effect = side_effect
print(m())
```

Results in:
```
Side effect func called
Original func called
ORIGINAL
```

Whilst the expected is:
```
Side effect func called
SIDE EFFECT
```



## TL;DR;

Indeed, `side_effect` seems broken when applied to an object with wraps and 
setting `return_value` kind of "fixes it".
I'd send a fix to add the check for `effect` unless michael.foord or someone 
else knows a good reason of why things are not like that. + tests

Noam Yorav-Raphael I am happy to send the patch if you don't have time :)

--

___
Python tracker 
<https://bugs.python.org/issue35330>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35330] When using mock to wrap an existing object, side_effect requires return_value

[Mario Corchero]

Mario Corchero  added the comment:

I'll get ready a PR with a good set of tests and the fix for the original 
issue. This is quite an interesting bug :)

--

___
Python tracker 
<https://bugs.python.org/issue35330>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3533] mac 10.4 buld of 3.0 --with-pydebug fails no __eprintf

[Mario Corchero]

Change by Mario Corchero :


--
pull_requests: +10215

___
Python tracker 
<https://bugs.python.org/issue3533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35330] When using mock to wrap an existing object, side_effect requires return_value

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +10234
stage: test needed -> patch review

___
Python tracker 
<https://bugs.python.org/issue35330>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17185] unittest mock create_autospec doesn't correctly replace mocksignature

[Mario Corchero]

Mario Corchero  added the comment:

Agree that it sounds reasonable to just set `__signature__` to tell `inspect` 
where to look at (thanks PEP 362 :P). Not an expert here though.

For the partials bug, I'll add Pablo as we were speaking today about something 
similar :) but that might be unrelated (though interesting and worth fixing!) 
from the issue Chris brought up.

--
nosy: +pablogsal

___
Python tracker 
<https://bugs.python.org/issue17185>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32299] unittest.mock.patch.dict.__enter__ should return the dict

[Mario Corchero]

Change by Mario Corchero :


--
pull_requests: +10296

___
Python tracker 
<https://bugs.python.org/issue32299>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35500] Align expected and actual calls on mock.assert_called_with error message

[Mario Corchero]

Mario Corchero  added the comment:

Makes sense!

I'd not align them though but that might be my view as I generally don't like 
aligning text like that.

Also if you feel that the exceptions read "weird" with the first sentence is 
empty, an option might be to say the calls don't match, to make it symmetric 
with the assert_calls message. Example:


Traceback (most recent call last):
  File "/tmp/bar.py", line 5, in 
m.assert_called_with(2, 3)
  File 
"/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", 
line 827, in assert_called_with
raise AssertionError(_error_message()) from cause
AssertionError: Expected call not found.
Expected call: mock(2, 3)
Actual call: mock(1, 2)

This way all error reports give you the issue in the first line of the message 
and further details in the next lines.

--

___
Python tracker 
<https://bugs.python.org/issue35500>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24928] mock.patch.dict spoils order of items in collections.OrderedDict

[Mario Corchero]

Mario Corchero  added the comment:

I would suggest applying the fix with the latest version in mind to keep the 
codebase clean. If you want to backport it to previous versions of Python you 
can do it manually through a PR targetting the right branch.
I think the process is to set up a label and then use 
https://github.com/python/core-workflow/tree/master/cherry_picker as I'd expect 
the tests to fail to apply the patch "manually.

Alternative 1: Might be easier is to send a patch that works in all version and 
another one that "modernises" it to python3.7. Basically, write all tests with 
OrderedDict and then just shoot an extra PR that converts that to a plain dict.

Alternative 2: This is only a problem on versions on versions < 3.6, isn't it? 
If so you might want to close this issue or just have a PR that targets the 3.5 
if worth backporting and/or PyPI's. (This is my conservative mind as you know I 
usually push for changing as less as possible hehe).

--

___
Python tracker 
<https://bugs.python.org/issue24928>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30189] SSL match_hostname does not accept IP Address

[Mario Viapiano]

New submission from Mario Viapiano:

I need this patch to be available in python 2.7.13
https://bugs.python.org/issue23239

--
components: Extension Modules
messages: 292468
nosy: emeve89
priority: normal
severity: normal
status: open
title: SSL match_hostname does not accept IP Address
type: enhancement
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue30189>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30541] Add restricted mocks to the python unittest mocking framework

[Mario Corchero]

New submission from Mario Corchero:

Define a way to disable the automatic generation of submocks when accessing an 
attribute of a mock which is not set.


Rationale:
Inspired by GMock RestrictedMock, it aims to allow the developer to declare a 
narrow interface to the mock that defines what the mocks allows to be called on.
The feature of mocks returning mocks by default is extremely useful but not 
always desired. Quite often you rely on it only at the time you are writing the 
test but you want it to be disabled at the time the mock is passed into your 
code.

It also prevents user errors when mocking incorrect paths or having typos when 
calling attributes/methods of the mock.
We have tried it internally in our company and it gives quite a nicer user 
experience for many use cases, specially for new users of mock as it helps out 
when you mock the wrong path.

Posible interfaces:
New Mock type, SeledMock which can be used instead of the "common" mock that 
has an attribute "sealed" which once set to true disables the dynamic 
generation of "submocks"


The final goal is to be able to write tests like:

>>> m = mock.Mock()  # or = mock.SealableMock()
>>> m.method1.return_value.attr1.method2.return_value = 1
>>> mock.seal(m)  # or mock.sealed = True

>>> m.method1().attr1.method2()  # This path has been declared above
# 1
>>> m.method1().attr2  # This was not defined so it is going to raise a 
>>> meaningful exception
# Exception: SealedMockAttributeAccess: mock.method1().attr2

--
components: Library (Lib)
messages: 294961
nosy: Mario Corchero, haypo
priority: normal
severity: normal
status: open
title: Add restricted mocks to the python unittest mocking framework
type: enhancement
versions: Python 3.7

___
Python tracker 
<http://bugs.python.org/issue30541>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30541] Add restricted mocks to the python unittest mocking framework

[Mario Corchero]

Mario Corchero added the comment:

Sample implementation using the new class:
https://github.com/mariocj89/cpython/commit/2f13963159e239de041cd68273b9fc4a2aa778cd

Sample implementation using the new function to seal existing mocks:
https://github.com/mariocj89/cpython/commit/9ba039e3996f4bf357d4827123e0b570d84f5bb6

Happy to submit a PR if the idea is accepted.

The only benefit I see from the using a separate class is that you will be able 
to do:

>>> m = mock.SealedMock()
>>> m.important_attr = 42
>>> m.freeflow_attribute = mock.Mock()
>>> mock.seal(m)

which will allow to define "subparts of the mock" without the sealing.

That said I still prefer the function implementation as it looks much nicer 
(credit to Victor Stinner for the idea)

--

___
Python tracker 
<http://bugs.python.org/issue30541>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30541] Add restricted mocks to the python unittest mocking framework

[Mario Corchero]

Changes by Mario Corchero :


--
pull_requests: +2003

___
Python tracker 
<http://bugs.python.org/issue30541>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35512] patch.dict resolves in_dict eagerly (should be late resolved)

[Mario Corchero]

Mario Corchero  added the comment:

Interesting, `patch` does resolve it when the patched function is called (see 
https://github.com/python/cpython/blob/175421b58cc97a2555e474f479f30a6c5d2250b0/Lib/unittest/mock.py#L1269)
 vs patch.dict that resolves it at the time the patcher is created - when 
decorating -  (see 
https://github.com/python/cpython/blob/175421b58cc97a2555e474f479f30a6c5d2250b0/Lib/unittest/mock.py#L1624).

An option might be to delay the resolution as done for patch, changing 
https://github.com/python/cpython/blob/175421b58cc97a2555e474f479f30a6c5d2250b0/Lib/unittest/mock.py#L1625
 to `self.in_dict_name = in_dict`

Example untested patch:

```
diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py
index 8f46050462..5328fda417 100644
--- a/Lib/unittest/mock.py
+++ b/Lib/unittest/mock.py
@@ -1620,9 +1620,7 @@ class _patch_dict(object):
 """

 def __init__(self, in_dict, values=(), clear=False, **kwargs):
-if isinstance(in_dict, str):
-in_dict = _importer(in_dict)
-self.in_dict = in_dict
+self.in_dict_name = in_dict
 # support any argument supported by dict(...) constructor
 self.values = dict(values)
 self.values.update(kwargs)
@@ -1649,7 +1647,7 @@ class _patch_dict(object):
 attr_value = getattr(klass, attr)
 if (attr.startswith(patch.TEST_PREFIX) and
  hasattr(attr_value, "__call__")):
-decorator = _patch_dict(self.in_dict, self.values, self.clear)
+decorator = _patch_dict(self.in_dict_name, self.values, 
self.clear)
 decorated = decorator(attr_value)
 setattr(klass, attr, decorated)
 return klass
@@ -1662,7 +1660,11 @@ class _patch_dict(object):

 def _patch_dict(self):
 values = self.values
-in_dict = self.in_dict
+if isinstance(self.in_dict_name, str):
+in_dict = _importer(self.in_dict_name)
+else:
+in_dict = self.in_dict_name
+self.in_dict = in_dict
```


> This seems to be not a problem with patch.object where redefining a class 
> later like dict seems to work correctly and maybe it's due to creating a new 
> class itself that updates the local to reference new class?

For patch, when you create a new class, the new one is patched as the name is 
resolved at the time the decorated function is executed, not when it is 
decorated. See:

```
$ cat t.py
from unittest import mock
import c

target = dict(a=1)

@mock.patch("c.A", "target", "updated")
def test_with_decorator():
print(f"target inside decorator : {A.target}")

def test_with_context_manager():
with mock.patch("c.A", "target", "updated"):
print(f"target inside context : {A.target}")

class A:
target = "changed"

c.A = A
test_with_decorator()
test_with_context_manager()
xarmariocj89 at DESKTOP-9B6VH3A in ~/workspace/cpython on master*
$ cat c.py
class A:
target = "original"
mariocj89 at DESKTOP-9B6VH3A in ~/workspace/cpython on master*
$ ./python ./t.py
target inside decorator : changed
target inside context : changed
```

If `patch` was implemented like `patch.dict`, you would see the first as 
"changed" as the reference to `c.A` would have been resolved when the decorator 
was run (before the re-definition of `A`).

About `patch.object`, it cannot be compared, as it grabs the name at the time 
you execute the decorator because you are not passing a string, but the actual 
object to patch.

--

___
Python tracker 
<https://bugs.python.org/issue35512>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35512] patch.dict resolves in_dict eagerly (should be late resolved)

[Mario Corchero]

Mario Corchero  added the comment:

Great, all yours :) I'll be happy to review.

--

___
Python tracker 
<https://bugs.python.org/issue35512>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26752] Mock(2.0.0).assert_has_calls() raise AssertionError in two same calls

[Mario Corchero]

Mario Corchero  added the comment:

Quite a tricky bug!

Indeed @xtreak the `_call_matcher` is using the `__init__` signature. I think 
this is a bug introduced in https://bugs.python.org/issue17015. There is a mix 
of the fact that spec in Mock also can accept classes (not instances) whilst 
spec requires the user to say whether what you are passing is a class or an 
instance. This gets messed up when validating the calls as at validation time 
it tries to match the signature as done in issue17015 (something that is useful 
for other cases as outlined in the issue).


You can see how this is clearly a bug with the following reproducer:

```
from unittest.mock import Mock, call
class X(object):
def __init__(self):pass
def foo(self, a):pass
x = Mock(spec=X)
x.foo(20)
x.assert_has_calls(x.mock_calls)
```


Using an instance (`X()`) of the class "hides" the issue, as the right 
signature is used for validation.


I am not sure if there is any easy fix here :/, but it is broken that the 
validation happens in different ways when a class is used in the spec (when 
calling it vs when validating it). Things "work" as if you use a spec of a 
class and then construct it, that passes fine as it does not get validated as 
attribute lookup and then there is no further validation.

Maybe something like this would work:

```
--- a/Lib/unittest/mock.py
+++ b/Lib/unittest/mock.py
@@ -384,8 +384,10 @@ class NonCallableMock(Base):
 def __init__(
 self, spec=None, wraps=None, name=None, spec_set=None,
 parent=None, _spec_state=None, _new_name='', _new_parent=None,
-_spec_as_instance=False, _eat_self=None, unsafe=False, **kwargs
+_spec_as_instance=None, _eat_self=None, unsafe=False, **kwargs
 ):
+if _spec_as_instance is None:
+_spec_as_instance = isinstance(spec, type) # We might need to play 
with eat_self as well here.
 if _new_parent is None:
 _new_parent = parent

@@ -2205,8 +2207,8 @@ def create_autospec(spec, spec_set=False, instance=False, 
_parent=None,
 elif spec is None:
 # None we mock with a normal mock without a spec
 _kwargs = {}
-if _kwargs and instance:
-_kwargs['_spec_as_instance'] = True
+if _kwargs:
+_kwargs['_spec_as_instance'] = instance
```

Basically, being explicit on auto_spec and inferring it on the creation of 
Mocks, but that might break some people who (probably badly) rely on the 
signature of the class.

This issue probably needs some further work.

--

___
Python tracker 
<https://bugs.python.org/issue26752>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36518] Avoid conflicts when pass arbitrary keyword arguments to Python function

[Mario Corchero]

Change by Mario Corchero :


--
nosy: +mariocj89

___
Python tracker 
<https://bugs.python.org/issue36518>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17013] Allow waiting on a mock

[Mario Corchero]

Mario Corchero  added the comment:

I think this is REALLY interesting!, there are many situations where this has 
been useful, it would greatly improve multithreading testing in Python.

Q? I see you inherit from Mock, should it inherit from MagicMock?
I'd say send the PR and the discussion can happen there about the 
implementation, seems there is consensus in this thread.

I would find it OK to start with `wait_until_called` if you want and can be 
discussed in another issue/PR if you don't have the time/think it might not be 
worth.

--

___
Python tracker 
<https://bugs.python.org/issue17013>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17013] Allow waiting on a mock

[Mario Corchero]

Mario Corchero  added the comment:

Kooning great! I would Add a test for the following (I think both fails with 
the proposed impl):

- The mock is called BEFORE calling wait_until_called_with
- I call the mock with arguments and then I call wait for call without 
arguments.
- using keyword arguments 

Also, I don’t have a great solution for it but it might be worth prefixing 
time-out with something in the wait_untill_called_with. In situations where the 
mocked object has a timeout parameter (which is a common argument for 
multithreaded apps).

--

___
Python tracker 
<https://bugs.python.org/issue17013>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36820] Captured exceptions are keeping user objects alive unnecessarily in the stdlib

[Mario Corchero]

New submission from Mario Corchero :

There are multiple places in the standard library where the code captures an 
exception and reraises it later (outside of the original except).

This is known to cause a cycle as saving the exception has a traceback that 
eventually points back to the exception, but it is especially problematic as it 
keeps alive the objects that the user has as well.

This can be reproduced with the following example:

```
import weakref

class A: pass

ref = None

def x():
global ref
cool_var = A()
ref = weakref.ref(cool_var)
assert ref()
try:
1/0
except Exception as e:
ee = e

try:
x()
except Exception:
pass

print(ref())
assert ref() is None
```

There are places in the standard library that try to get around this. See 
https://github.com/python/cpython/commit/acb9fa79fa6453c2bbe3ccfc9cad2837feb90093
 as an example. This change did not include the exception path though, when the 
`err` variable is raised, the issue is still present.

This issue is to fix the instances found in socket.py, codeop.py and dyld.py. 
By either setting the variable to None to remove the name or if it needs to be 
reraised by using a finally to delete it. This is basically the same that a try 
except would do, which automagically adds a finally to delete the local name 
used in the `except X as y`.
There was a discussion with twouters,pablogsal,barry about potentially adding 
something extra to the exception handling to try to clean this up but it was 
suggested the best approach is for the user to handle it and maybe a linter to 
potentially flag it, as there might also be multiple references to the 
exception being captured. It was also proposed to just change the situations 
where this happens in the standard library to remove the name if possible.

Note that eventually those objects will just be collected by the gc, it is just 
an improvement.

I'm preparing a PR for those three modules.

--
components: Library (Lib)
messages: 341620
nosy: mariocj89, pablogsal, twouters
priority: low
severity: normal
status: open
title: Captured exceptions are keeping user objects alive unnecessarily in the 
stdlib
type: enhancement
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue36820>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36820] Captured exceptions are keeping user objects alive unnecessarily in the stdlib

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +13048
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue36820>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36848] autospec fails with AttributeError when mocked class has __signature__ non-writeable

[Mario Corchero]

Mario Corchero  added the comment:

Agree, for the general case I think it makes sense that this is fixed in pyside.
Hijacking __signature__ python-wide sounds like it can cause other issues, if 
they need such a hack I would suggest finding a way (in pyside) to allow for 
mock to work.

I wonder though if this change in __signature__ will affect in any way any C 
extension that might not allow setting __signature__ if not set as a lack of 
__dict__ (I have not managed to find any issue with small examples though).

--

___
Python tracker 
<https://bugs.python.org/issue36848>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32206] Run modules with pdb

[Mario Corchero]

Mario Corchero  added the comment:

Hi Jason, thanks a lot! 

I’ll have a look to the bug you reported on Monday. Out for holidays atm ^^

--

___
Python tracker 
<https://bugs.python.org/issue32206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32691] "pdb -m " sets __main__.__package__ incorrectly

[Mario Corchero]

Mario Corchero  added the comment:

OK, just managed to reproduce it.

This appears only when you run a python script as a module. Running a module 
with  __main__ does not show the issue.

Will get a patch ready

--

___
Python tracker 
<https://bugs.python.org/issue32691>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32206] Run modules with pdb

[Mario Corchero]

Change by Mario Corchero :


--
pull_requests: +5301

___
Python tracker 
<https://bugs.python.org/issue32206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32691] "pdb -m " sets __main__.__package__ incorrectly

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +5302
stage: test needed -> patch review

___
Python tracker 
<https://bugs.python.org/issue32691>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32691] "pdb -m " sets __main__.__package__ incorrectly

[Mario Corchero]

Mario Corchero  added the comment:

Sent a PR for the fix. I'll update PRs for trace. profile does not need it

Thanks a lot for bringing it up Jason!

--

___
Python tracker 
<https://bugs.python.org/issue32691>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32206] Run modules with pdb

[Mario Corchero]

Change by Mario Corchero :


--
pull_requests:  -5301

___
Python tracker 
<https://bugs.python.org/issue32206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32206] Run modules with pdb

[Mario Corchero]

Change by Mario Corchero :


--
pull_requests: +5321

___
Python tracker 
<https://bugs.python.org/issue32206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32206] Run modules with pdb

[Mario Corchero]

Change by Mario Corchero :


--
pull_requests: +5321, 5322

___
Python tracker 
<https://bugs.python.org/issue32206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32821] Add snippet on how to configure a "split" stream for console

[Mario Corchero]

New submission from Mario Corchero :

As discussed in python-ideas, it would be good to have a recipe on how to 
configure the logging stack to be able to log ``ERROR`` and above to stderr and 
``INFO`` and below to stdout.

It was proposed to add it into the cookbook rather than on the standard 
library. An alternative proposal was a "ConsoleHandler" that would do this by 
default. 
(https://github.com/mariocj89/cpython/commit/501cfcd0f4cad1e04d87b89784988c52a77a80ad)

--
assignee: docs@python
components: Documentation
messages: 312003
nosy: docs@python, mariocj89
priority: normal
severity: normal
status: open
title: Add snippet on how to configure a "split" stream for console
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue32821>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32821] Add snippet on how to configure a "split" stream for console

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +5433
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue32821>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31800] datetime.strptime: Support for parsing offsets with a colon

[Mario Corchero]

New submission from Mario Corchero :

Currently, datetime.strptime does not support parsing utc offsets that include 
a colon. "+" is parsed without issues whilst it fails with "+00:00".

"+NN:NN" is not only ISO8601 valid but also the way the offset is presented to 
the user when using .isoformat on a datetime with a timezone/offset.

This lead to the users needing to go to external libraries like dateutil or 
iso8601 just to be able to parse the datetime encoded in strings that 
"datetime" produces.

Even if a long-term goal would be to provide a way to parse any isoformatted 
string this issue just aims to address the problem that the %z parsing 
presents. This already unblocks users from parsing datetime object serialized 
with isoformat.

With this change, the following will just work:

>>> import datetime as dt
>>> iso_fmt = '%Y-%m-%dT%H:%M:%S%z'
>>> d = dt.datetime.strptime('2004-01-01T10:10:10+05:00', iso_fmt)

*'2004-01-01T10:10:10+05:00' is a sample string generated via 
datetime.isoformat()

Other options like having a new %:z was proposed but having just %z seems much 
simpler for the user.



Note: There has been already conversations about adding support on datetime to 
parse any ISO-formatted string. This is a more simplistic approach. We might be 
able to get to that situation after this patch, but this aims just to unblock 
us.

Related:
http://www.loc.gov/standards/datetime/iso-tc154-wg5_n0039_iso_wd_8601-2_2016-02-16.pdf
https://mail.python.org/pipermail/python-ideas/2014-March/027018.html
https://bugs.python.org/issue15873

--
components: Library (Lib)
messages: 304486
nosy: mariocj89
priority: normal
severity: normal
status: open
title: datetime.strptime: Support for parsing offsets with a colon
type: enhancement
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue31800>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31800] datetime.strptime: Support for parsing offsets with a colon

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +3989
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue31800>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31800] datetime.strptime: Support for parsing offsets with a colon

[Mario Corchero]

Mario Corchero  added the comment:

Yep, http://man7.org/linux/man-pages/man3/strptime.3.html does support it even 
if it might look asymetrical.

Example:

   struct tm tm;
   char buf[255];

   memset(&tm, 0, sizeof(struct tm));
   strptime("+00:00", "%z", &tm);
   strftime(buf, sizeof(buf), "%z", &tm);
   puts(buf); // Will print +
   exit(EXIT_SUCCESS);

Martin do you want me to "cleanup" the PR, add docs, news entry, etc?

--

___
Python tracker 
<https://bugs.python.org/issue31800>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24954] No way to generate or parse timezone as produced by datetime.isoformat()

[Mario Corchero]

Change by Mario Corchero :


--
nosy: +mariocj89

___
Python tracker 
<https://bugs.python.org/issue24954>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24954] No way to generate or parse timezone as produced by datetime.isoformat()

[Mario Corchero]

Mario Corchero  added the comment:

Wrote https://bugs.python.org/issue31800 without realising this issue was open 
(Thanks for bringing it up Martin Panter).

issue31800 basically just adds the ability to parse NN:NN to the already 
existing python isoformat function when %z is specified.
Rather than adding "%:z".

As available as well in linux strptime: 
http://man7.org/linux/man-pages/man3/strptime.3.html

I'd really like to see a way to parse isoformatted dates and this is the only 
thing in the middle. Happy to continue with the other issue/PR or help out here 
if needed.

Thanks! ^^

--

___
Python tracker 
<https://bugs.python.org/issue24954>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31454] Include "import as" in tutorial

[Mario Corchero]

Change by Mario Corchero :


--
nosy: +mariocj89

___
Python tracker 
<https://bugs.python.org/issue31454>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31454] Include "import as" in tutorial

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +4012
stage: needs patch -> patch review

___
Python tracker 
<https://bugs.python.org/issue31454>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31800] datetime.strptime: Support for parsing offsets with a colon

[Mario Corchero]

Mario Corchero  added the comment:

As a note

Seems support for the ":" was added in 2015 for glibc:
http://code.metager.de/source/xref/gnu/glibc/time/strptime_l.c#765

Commit e952e1df

Before that, it basically just ignores the minutes.

--

___
Python tracker 
<https://bugs.python.org/issue31800>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31800] datetime.strptime: Support for parsing offsets with a colon

[Mario Corchero]

Mario Corchero  added the comment:

I have a patch to add 'Z' support as well if we are interested in making it the 
same as it glibc does. (as it supports it as well)

--

___
Python tracker 
<https://bugs.python.org/issue31800>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30548] typo in documentation for create_autospec

[Mario Corchero]

Mario Corchero  added the comment:

I've always understood instance as a way to say "I am passing this class but I 
want to force the autospec on the instance"

For example, given

```
class X:
def __init__(self):
raise
```

You can do `unittest.mock.create_autospec(X, instance=True)` to set a spec of 
the instance rather than the class.

Also quite often you do autospec on a class but you want the interface of the 
instance. This parameter allows you to do so.

Basically, `unittest.mock.create_autospec(X, instance=True)` will produce a non 
callable mock.

I think the docs are correct, maybe misleading

--
nosy: +mariocj89

___
Python tracker 
<https://bugs.python.org/issue30548>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30699] Misleading class names in datetime.tzinfo usage examples

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +4253
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue30699>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com