Re: [Python-Dev] Tests failing on Windows with TESTFN
On 29/07/2018 02:04, Jeremy Kloth wrote: On Sat, Jul 28, 2018 at 6:43 PM Brett Cannon wrote: If Windows doesn't clean up its temp directory on a regular basis then that doesn't suggest to me not to use tempfile, but instead that the use of tempfile still needs to clean up after itself. And if there is a lacking feature in tempfile then we should add it instead of a avoiding the module. Mind you, this is mentioned in the confines of the test harness where just about anything can happen (and usually does!). Something that cannot be coded against using just tempfile is cleanup on process abort. The per-process-directory approach handles this case. I would think it is desired to have no leftovers after running the test harness (especially in regards to the buildbots). Now, I'm not sure the exact cause of all of the leftovers in the TEMP directory, but it is definitely something that is currently happening (and shouldn't be). It is not exactly the easiest of tasks to track the file usage of every test in the test suite. It is certainly easier to replace usages of os.unlink with test.support.unlink within the test suite. In the interests of trying to keep a focus to the changes I'm making, I propose to start again by, as you suggest, making use of test.support.unlink where it's not currently used. From the evidence I don't believe that will solve every problem I'm seeing but it should certainly reduce them. I do there there's mileage in a wider change to revamp the test suite's naming and cleanup of temporary files but I'm very wary of trying to undertake what would undoubtedly be a sprawling and probably contentious change. TJG ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On 28/07/2018 22:17, Jeremy Kloth wrote: On Sat, Jul 28, 2018 at 11:20 AM Tim Golden wrote: Although things have moved on since that discussion and test.support.unlink has grown some extra legs, all it's done really is to push the bump along the carpet for a bit. I've got a newly-installed Win10 machine with the typical MS Antivirus & TortoiseGitCache vying for locks with every other process. I've just done a full test run: python -mtest -j0 -v > test.log I, for one, would like to see that log. The issues you are have are fairly unique. Just check out the buildbot status page. I know that some of the workers are somewhat limited, but my worker (https://buildbot.python.org/all/#/workers/12) is running on dedicated hardware. Before https://bugs.python.org/issue15496 was applied, the errors you describe were indeed happening, but no longer. For an example: http://tjg.org.uk/test.log Thinkpad T420, 4Gb, i5, SSD Recently rebuilt and reinstalled: Win10, VS2017, TortoiseGit, standard Windows Antimalware, usual developer tools. That particular run was done with the laptop unattended (ie nothing else going on at the front end). But the problem is certainly not specific to this laptop. TJG ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Testing C API
Currently C API is not completely covered by tests. Tests for particular parts of C API are scattered through different files. There are files completely purposed for testing C API (like test_capi.py, test_getargs2.py), there are classes (usually having "CAPI" in the name) in different files for testing C API specific for unicode, marshal. Argument parsing tests are split between two files, test_capi.py, test_getargs2.py. I need to add new tests for new features, and I'm going to add new tests for existing C API. But first I'm going to reorganize tests. Add a new directory Lib/test/test_capi, and move all C API tests into it, grouped by function prefixes. test_getargs.py for testing PyArg_*(), test_unicode.py for testing PyUnicode_*(), etc. Tests that use the _testcapi module, but don't test specific C API, will left on place. The benefit is that it will be easier to run all C API tests at once, and only them, and it will be clearer what C API is covered by tests. The disadvantage is that you will need to run several files for testing marshal for example. What are your thoughts? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On 29Jul2018 0958, Tim Golden wrote: In the interests of trying to keep a focus to the changes I'm making, I propose to start again by, as you suggest, making use of test.support.unlink where it's not currently used. From the evidence I don't believe that will solve every problem I'm seeing but it should certainly reduce them. One additional thing that may help (if support.unlink doesn't already do it) is to rename the file before deleting it. Renames are always possible even with open handles, and then you can create a new file at the original name. Cheers, Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Testing C API
On 29Jul2018 1253, Serhiy Storchaka wrote: The benefit is that it will be easier to run all C API tests at once, and only them, and it will be clearer what C API is covered by tests. The disadvantage is that you will need to run several files for testing marshal for example. Can we make the regular tests import and also run the related C API tests? So that a normal run wouldn't normally include the entire C API test directory, but would include test classes in the related Python test modules? (Maybe there's a way to decorate the test classes for this?) I agree with the intent, but also think that's quite a disadvantage. It would be good to avoid it. Cheers, Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Testing C API
29.07.18 15:39, Steve Dower пише: On 29Jul2018 1253, Serhiy Storchaka wrote: The benefit is that it will be easier to run all C API tests at once, and only them, and it will be clearer what C API is covered by tests. The disadvantage is that you will need to run several files for testing marshal for example. Can we make the regular tests import and also run the related C API tests? So that a normal run wouldn't normally include the entire C API test directory, but would include test classes in the related Python test modules? (Maybe there's a way to decorate the test classes for this?) There are many ways of running tests: ./python -m test test_capi ./python -m test.test_capi ./python -m unittest test.test_capi ./python -m unittest discover Lib/test/test_capi/ They need different solutions for making them disabled by default. Seems that the simplest way is to move test_capi out of the test directory. But I think that in any case this will complicate testing code. I agree with the intent, but also think that's quite a disadvantage. It would be good to avoid it. Actually this disadvantage is not very large. There are not much C API tests for now. Testing unicode requires running not just test_unicode, but test_codecs, test_codeccallbacks, test_format, and yet few test. test_bytes itself is a mess, it needs significant rewriting. Marshal C API is outdated (it is based on FILE*), it is mostly unused in CPython. In any case Python tests should be enough for testing Python API. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On Sun, Jul 29, 2018 at 9:13 AM, Tim Golden wrote:
>
> For an example:
>
> http://tjg.org.uk/test.log
>
> Thinkpad T420, 4Gb, i5, SSD
>
> Recently rebuilt and reinstalled: Win10, VS2017, TortoiseGit, standard
> Windows Antimalware, usual developer tools. That particular run was done
> with the laptop unattended (ie nothing else going on at the front end).
> But the problem is certainly not specific to this laptop.
On my last run I had one test directory that wasn't removed properly,
but nothing like the flood of EACCES and ERROR_ACCES_DENIED errors you
have in that log. Then again, I had Defender disabled by policy. I'll
enable it and add exceptions for my source and build directories, and
see how it goes.
It would be nice if OSError instances always captured the last Windows
error and NT status values when instantiated. We have no guarantees
that these values are valid, but in many contexts they are. In the
case of a test log, it would certainly help to clarify errors without
having to individually investigate each one.
For example, trying to open a directory as a file is a common error,
but all Python tells us on Windows is that it failed with EACCES. In
this case the last Windows error is ERROR_ACCESS_DENIED, which doesn't
help, but the last NT status code is STATUS_FILE_IS_A_DIRECTORY
(0xc0ba).
Here's a file opener that adds last_winerror and last_ntstatus values.
import os
ntdll = ctypes.WinDLL('ntdll')
kernel32 = ctypes.WinDLL('kernel32')
def nt_opener(path, flags):
try:
return os.open(path, flags)
except OSError as e:
last_ntstatus = ntdll.RtlGetLastNtStatus()
last_winerror = kernel32.GetLastError()
e.last_ntstatus = last_ntstatus & 2**32 - 1
e.last_winerror = (last_winerror if e.winerror is None
else e.winerror)
if e.errno is not None or e.winerror is not None:
# hack the last error/status into the error message
e.strerror = '[Last NtStatus {:#08x}] {}'.format(
e.last_ntstatus, e.strerror or '')
if e.winerror is None:
e.strerror = '[Last WinError {}] {}'.format(
e.last_winerror, e.strerror or '')
raise e from None
Opening a directory as a file:
>>> open('C:/Windows', opener=nt_opener)
Traceback (most recent call last):
File "", line 1, in
File "", line 17, in nt_opener
File "", line 3, in nt_opener
PermissionError: [Errno 13] [Last WinError 5] [Last NtStatus 0xc0ba]
Permission denied: 'C:/Windows'
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On Sun, Jul 29, 2018 at 3:13 AM Tim Golden wrote:
> For an example:
>
> http://tjg.org.uk/test.log
Thank you! After inspecting all the errors, it does seem that they
are ALL caused by "bare" os.unlink/rmdir calls. So it seems that a
massive undertaking of ferreting out these locations and replacing
them with their support equivalents would be needed to have a passing
test suite for you.
Unfortunately, test_mailbox is failing due to the same fate, but
within the stdlib itself. The fix for that will probably need to be a
rename/delete dance.
diff --git a/Lib/mailbox.py b/Lib/mailbox.py
index 056251d..23662f2 100644
--- a/Lib/mailbox.py
+++ b/Lib/mailbox.py
@@ -701,8 +701,10 @@ class _singlefileMailbox(Mailbox):
try:
os.rename(new_file.name, self._path)
except FileExistsError:
-os.remove(self._path)
+temp_name = _create_temporary_name(self._path)
+os.rename(self._path, temp_name)
os.rename(new_file.name, self._path)
+os.remove(temp_name)
self._file = open(self._path, 'rb+')
self._toc = new_toc
self._pending = False
@@ -2112,11 +2114,14 @@ def _create_carefully(path):
finally:
os.close(fd)
+def _create_temporary_name(path):
+"""Create a temp filename based on path."""
+return '%s.%s.%s.%s' % (path, int(time.time()), socket.gethostname(),
+os.getpid())
+
def _create_temporary(path):
"""Create a temp file based on path and open for reading and writing."""
-return _create_carefully('%s.%s.%s.%s' % (path, int(time.time()),
- socket.gethostname(),
- os.getpid()))
+return _create_carefully(_create_temporary_name(path))
def _sync_flush(f):
"""Ensure changes to file f are physically on disk."""
--
Jeremy Kloth
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On Sun, Jul 29, 2018 at 12:35 PM, Steve Dower wrote: > > One additional thing that may help (if support.unlink doesn't already do it) > is to rename the file before deleting it. Renames are always possible even > with open handles, and then you can create a new file at the original name. Renaming open files typically fails with a sharing violation (32). Most programs open files with read and write sharing but not delete sharing. This applies to Python, except temporary files (i.e. os.O_TEMPORARY) do share delete access. Renaming a file is effectively adding a new link and deleting the old link, so it requires opening the file with delete access. Also, renaming a directory that has open files in the tree fails with access denied (5). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On Sun, Jul 29, 2018 at 2:21 PM, Jeremy Kloth wrote: > > try: > os.rename(new_file.name, self._path) > except FileExistsError: > -os.remove(self._path) > +temp_name = _create_temporary_name(self._path) > +os.rename(self._path, temp_name) > os.rename(new_file.name, self._path) > +os.remove(temp_name) This should call os.replace to allow the file system to replace the existing file. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On 29/07/2018 15:21, Jeremy Kloth wrote: On Sun, Jul 29, 2018 at 3:13 AM Tim Golden wrote: For an example: http://tjg.org.uk/test.log Thank you! After inspecting all the errors, it does seem that they are ALL caused by "bare" os.unlink/rmdir calls. So it seems that a massive undertaking of ferreting out these locations and replacing them with their support equivalents would be needed to have a passing test suite for you. Thanks for checking. As I mentioned elsewhere in this thread, I propose to go through and apply support.unlink where necessary. It won't make things any worse and will hopefully improve in some areas. For test_mailbox I've experimentally implemented a hybrid tempfile / local directory solution. ie I've created a new file on each run, but only within the python_ folder which already exists. As long as the directory cleans up there should be no leftovers. That's certainly helped although my re-run harness has provoked at least one error. TJG ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Using Python on a fork-less POSIX-like OS
Hello Python list, I intend to cross-compile Python v3.6.6 to Threos ( https://threos.io ) operating system. Threos is supports a quite large set from POSIX and C89/C99. Unfortunately, Threos lacks fork(2), but provides posix_spawn(3) instead. I already made some local changes in posixmodule.c to compile due to some features are detected as present but actually not supported, like HAVE_FORK -- I blame autotools for this :-). I don't know, however, whether the Python shall cross-compile without issues. My question is that the _posixsubprocess.c can be prepared to use posix_spawn(3) instead of fork(2)? Maybe the UNIX/Linux version can also benefit from it, see: https://salsa.debian.org/ruby-team/ruby-posix-spawn Best regards, Aron ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On Sun, Jul 29, 2018 at 9:34 AM Tim Golden wrote: > For test_mailbox I've experimentally implemented a hybrid tempfile / > local directory solution. ie I've created a new file on each run, but > only within the python_ folder which already exists. As long as the > directory cleans up there should be no leftovers. That's certainly > helped although my re-run harness has provoked at least one error. As Eryk noted, the fix for mailbox.py (yes, the stdlib needs fixing in this case) is quite simple: diff --git a/Lib/mailbox.py b/Lib/mailbox.py index 056251d..eb85df1 100644 --- a/Lib/mailbox.py +++ b/Lib/mailbox.py @@ -701,8 +701,7 @@ class _singlefileMailbox(Mailbox): try: os.rename(new_file.name, self._path) except FileExistsError: -os.remove(self._path) -os.rename(new_file.name, self._path) +os.replace(new_file.name, self._path) self._file = open(self._path, 'rb+') self._toc = new_toc self._pending = False -- Jeremy Kloth ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using Python on a fork-less POSIX-like OS
On Sun, Jul 29, 2018 at 5:44 PM, Barath Aron wrote: > My question is that the _posixsubprocess.c can be prepared to use > posix_spawn(3) instead of fork(2)? Maybe the UNIX/Linux version can also > benefit from it, see: > https://salsa.debian.org/ruby-team/ruby-posix-spawn There is an open issue to add os.posix_spawn() at https://bugs.python.org/issue20104 --Berker ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
If the problem is AV scanners, then they should be opening them properly for this (and since the delete is not failing but is being deferred, I assume it's allowing deletes). If the problem is elsewhere in our code base then we have a different bug. Top-posted from my Windows 10 phone From: eryk sun Sent: Sunday, 29 July 2018 15:28 To: [email protected] Subject: Re: [Python-Dev] Tests failing on Windows with TESTFN On Sun, Jul 29, 2018 at 12:35 PM, Steve Dower wrote: > > One additional thing that may help (if support.unlink doesn't already do it) > is to rename the file before deleting it. Renames are always possible even > with open handles, and then you can create a new file at the original name. Renaming open files typically fails with a sharing violation (32). Most programs open files with read and write sharing but not delete sharing. This applies to Python, except temporary files (i.e. os.O_TEMPORARY) do share delete access. Renaming a file is effectively adding a new link and deleting the old link, so it requires opening the file with delete access. Also, renaming a directory that has open files in the tree fails with access denied (5). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Testing C API
On Sun, Jul 29, 2018, 06:44 Serhiy Storchaka, wrote: > 29.07.18 15:39, Steve Dower пише: > > On 29Jul2018 1253, Serhiy Storchaka wrote: > >> The benefit is that it will be easier to run all C API tests at once, > >> and only them, and it will be clearer what C API is covered by tests. > >> The disadvantage is that you will need to run several files for > >> testing marshal for example. > > > > Can we make the regular tests import and also run the related C API > > tests? So that a normal run wouldn't normally include the entire C API > > test directory, but would include test classes in the related Python > > test modules? (Maybe there's a way to decorate the test classes for > this?) > > There are many ways of running tests: > > ./python -m test test_capi > ./python -m test.test_capi > ./python -m unittest test.test_capi > ./python -m unittest discover Lib/test/test_capi/ > > They need different solutions for making them disabled by default. Seems > that the simplest way is to move test_capi out of the test directory. > But I think that in any case this will complicate testing code. > > > I agree with the intent, but also think that's quite a disadvantage. It > > would be good to avoid it. > > Actually this disadvantage is not very large. There are not much C API > tests for now. For now, but if are successful, Serhiy, there will be a lot more tests. 😉 Testing unicode requires running not just test_unicode, > but test_codecs, test_codeccallbacks, test_format, and yet few test. > test_bytes itself is a mess, it needs significant rewriting. Marshal C > API is outdated (it is based on FILE*), it is mostly unused in CPython. > In any case Python tests should be enough for testing Python API. > I think it depends on how you mentally group tests. Do you want to run all tests relating to marshal when you run test_marshal, or should you run test_marshal just for the Python code and test_capi.test_marshal for just the C API depending on what you changed? One perk of putting the C API tests in under test_capi is it makes it easier for other implementations to ignore those tests if they want to (although I'm assuming all the tests will also be marked as appropriate as CPython-specific). > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Let's change to C API!
Hi, I just sent an email to the capi-sig mailing list. Since this mailing list was idle for months, I copy my email here to get a wider audience. But if possible, I would prefer that you join me on capi-sig to reply ;-) -- Hi, Last year, I gave a talk at the Language Summit (during Pycon) to explain that CPython should become 2x faster to remain competitive. IMHO all attempts to optimize Python (CPython forks) have failed because they have been blocked by the C API which implies strict constraints. I started to write a proposal to change the C API to hide implementation details, to prepare CPython for future changes. It allows to experimental optimization ideas without loosing support for C extensions. C extensions are a large part of the Python success. They are also the reason why PyPy didn't replace CPython yet. PyPy cpyext remains slower than CPython because PyPy has to mimick CPython which adds a significant overhead (even if PyPy developers are working *hard* to optimize it). I created a new to discuss how to introduce backward incompatible changes in the C API without breaking all C extensions: http://pythoncapi.readthedocs.io/ The source can be found at: https://github.com/vstinner/pythoncapi/ I would like to create a team of people who want to work on this project: CPython, PyPy, Cython and anyone who depends on the C API. Contact me in private if you want to be added to the GitHub project. I propose to discuss on the capi-sig mailing list since I would like to involve people from various projects, and I don't want to bother you with the high traffic of python-dev. Victor PS: I added some people as BCC ;-) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using Python on a fork-less POSIX-like OS
On 07/29/2018 06:02 PM, Berker Peksağ wrote: There is an open issue to add os.posix_spawn() at https://bugs.python.org/issue20104 Seems promising, but 3.7 does not support it. And I don't see whether Python will work without fork(). - bpo-20104: Expose posix_spawn as a low level API in the os module. (removed before 3.7.0rc1) --Berker Aron ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's change to C API!
My first impression is that making things faster and hiding implementation details in the ABI are contrary goals. I agree with hiding implementation details in the API but not in the ABI. For example, you mention that you want to make Py_INCREF() a function call instead of a macro. But since Py_INCREF is very common, I would guess that this would make performance worse (not by much maybe but surely measurable). Jeroen. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's change to C API!
2018-07-29 23:41 GMT+02:00 Jeroen Demeyer : > For example, you mention that you want to make Py_INCREF() a function call > instead of a macro. But since Py_INCREF is very common, I would guess that > this would make performance worse (not by much maybe but surely measurable). For the very specific case of Py_INCREF(), yes, I agree that performance is an issue. But I don't see how I would hide completely the PyObject structure without converting Py_INCREF() macro with a function call. (I have reasons to want to hide everything, explained in the project.) The open question is if the cost of using function calls for Py_INCREF/DECREF versus the benefit of having the ability to modify deeply CPython internals. I'm not sure that it's worth to bet at this point, it's too early, and we can decide that later. Moreover, it's also possible to keep Py_INCREF() as a macro in the "backward compatible" mode, but require a function call in the mode which hides all implementation details (the one where you can experiment deep CPython internals changes). Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Let's change to C API!
On Mon, Jul 30, 2018 at 10:46 AM, Victor Stinner wrote: > 2018-07-29 23:41 GMT+02:00 Jeroen Demeyer : >> For example, you mention that you want to make Py_INCREF() a function call >> instead of a macro. But since Py_INCREF is very common, I would guess that >> this would make performance worse (not by much maybe but surely measurable). > > For the very specific case of Py_INCREF(), yes, I agree that > performance is an issue. But I don't see how I would hide completely > the PyObject structure without converting Py_INCREF() macro with a > function call. (I have reasons to want to hide everything, explained > in the project.) > > The open question is if the cost of using function calls for > Py_INCREF/DECREF versus the benefit of having the ability to modify > deeply CPython internals. > > I'm not sure that it's worth to bet at this point, it's too early, and > we can decide that later. Moreover, it's also possible to keep > Py_INCREF() as a macro in the "backward compatible" mode, but require > a function call in the mode which hides all implementation details > (the one where you can experiment deep CPython internals changes). > If the macro and function are absolutely 100% compatible, it would be possible to set compilation to use the function by default, and have a directive that switches to using the macro. It'd improve performance at the price of locking you to the exact CPython build. So within CPython itself, there'd be no performance cost (ergo if you mess with the internals, you have to recompile all of CPython), most extension libraries would pay a small (probably immeasurable) price for compatibility, and a small handful could opt to improve performance at the price of breaking if anything changes. ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Testing C API
> On Jul 29, 2018, at 4:53 AM, Serhiy Storchaka wrote: > > The benefit is that it will be easier to run all C API tests at once, and > only them, and it will be clearer what C API is covered by tests. The > disadvantage is that you will need to run several files for testing marshal > for example. > > What are your thoughts? I prefer the current organization that keeps the various tests together with the category being tested. I almost never need to run the C API tests all at once, but I do need to see all the tests for an object in one place. When maintaining something like marshal, it would be easy to miss some of the tests if they are in a separate file. IMO, the proposed change would hinder future maintenance and fly in the face of our traditional code organization. Raymond ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
