[Python-Dev] Rename Include/internals/ to Include/pycore/
Hi, Python C API has no strict separation between the 3 levels of API: * core: Py_BUILD_CORE define * stable: Py_LIMITED_API define (it has a value) * regular: no define IMHO the current design of header files is done backward: by default, everything is public. To exclude an API from core or stable, "#ifdef Py_BUILD_CORE" and "#ifndef Py_LIMITED_API" are used. This design caused issues in the past: functions, variables or something else exposed whereas they were supposed to be "private". I propose a practical solution for that: Include/*.h files would only be be public API. The "core" API would live in a new subdirectory: Include/pycore/*.h. Moreover, files of this subdirectory would have the prefix "pycore_". For example, Include/objimpl.h would have a twin: Include/pycore/pycore_objimpl.h which extend the public API with "core" APIs. I also propose to automatically load the twin: Include/objimpl.h would load Include/pycore/pycore_objimpl.h if Py_BUILD_CORE is defined: #ifdef Py_BUILD_CORE # include "pycore/pycore_objimpl.h" #endif Only in some rare cases, you would have to explicitly use: #include "pycore/pycore_pygetopt.h". This header is fully private, there is no public header in Include/pygetopt.h. Or maybe we should modify Include/Python.h to also include "pycore/pycore_pygetopt.h" if Py_BUILD_CORE is defined? Well, that's just a detail. First milestone: * Create Include/pycore/ * Move Py_BUILD_CORE specific code into Include/pycore/pycore_*.h * Automatically include pycore files from Include/*.h files (#ifdef Py_BUILD_CORE) Second milestone: * Find a solution for Py_LIMITED_API :-) Backward compatibility? The overall change is fully backward compatible. The default API doesn't change. C code (using header fles) doesn't have to be changed. Only a specific kinds of applications like debugger may have to be modified if they really have to access the "core" API, the "most private" API. Honestly, today it's unclear to me if this API can technically be used outside CPython. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rename Include/internals/ to Include/pycore/
Oh, I forgot to add a reference to the bugs.python.org issue and my pull request! * https://bugs.python.org/issue35081 * https://github.com/python/cpython/pull/10143 My PR more or less implements the first milestone of my plan (Py_BUILD_CORE): it creates Include/pycore/. Victor Le dim. 28 oct. 2018 à 17:20, Victor Stinner a écrit : > > Hi, > > Python C API has no strict separation between the 3 levels of API: > > * core: Py_BUILD_CORE define > * stable: Py_LIMITED_API define (it has a value) > * regular: no define > > IMHO the current design of header files is done backward: by default, > everything is public. To exclude an API from core or stable, "#ifdef > Py_BUILD_CORE" and "#ifndef Py_LIMITED_API" are used. This design > caused issues in the past: functions, variables or something else > exposed whereas they were supposed to be "private". > > I propose a practical solution for that: Include/*.h files would only > be be public API. The "core" API would live in a new subdirectory: > Include/pycore/*.h. Moreover, files of this subdirectory would have > the prefix "pycore_". For example, Include/objimpl.h would have a > twin: Include/pycore/pycore_objimpl.h which extend the public API with > "core" APIs. > > I also propose to automatically load the twin: Include/objimpl.h would > load Include/pycore/pycore_objimpl.h if Py_BUILD_CORE is defined: > > #ifdef Py_BUILD_CORE > # include "pycore/pycore_objimpl.h" > #endif > > Only in some rare cases, you would have to explicitly use: #include > "pycore/pycore_pygetopt.h". This header is fully private, there is no > public header in Include/pygetopt.h. Or maybe we should modify > Include/Python.h to also include "pycore/pycore_pygetopt.h" if > Py_BUILD_CORE is defined? Well, that's just a detail. > > First milestone: > > * Create Include/pycore/ > * Move Py_BUILD_CORE specific code into Include/pycore/pycore_*.h > * Automatically include pycore files from Include/*.h files (#ifdef > Py_BUILD_CORE) > > Second milestone: > > * Find a solution for Py_LIMITED_API :-) > > Backward compatibility? The overall change is fully backward > compatible. The default API doesn't change. C code (using header fles) > doesn't have to be changed. > > Only a specific kinds of applications like debugger may have to be > modified if they really have to access the "core" API, the "most > private" API. Honestly, today it's unclear to me if this API can > technically be used outside CPython. > > Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rename Include/internals/ to Include/pycore/
On Sun, Oct 28, 2018, at 09:20, Victor Stinner wrote: > Hi, > > Python C API has no strict separation between the 3 levels of API: > > * core: Py_BUILD_CORE define > * stable: Py_LIMITED_API define (it has a value) > * regular: no define > > IMHO the current design of header files is done backward: by default, > everything is public. To exclude an API from core or stable, "#ifdef > Py_BUILD_CORE" and "#ifndef Py_LIMITED_API" are used. This design > caused issues in the past: functions, variables or something else > exposed whereas they were supposed to be "private". > > I propose a practical solution for that: Include/*.h files would only > be be public API. The "core" API would live in a new subdirectory: > Include/pycore/*.h. Moreover, files of this subdirectory would have > the prefix "pycore_". For example, Include/objimpl.h would have a > twin: Include/pycore/pycore_objimpl.h which extend the public API with > "core" APIs. > > I also propose to automatically load the twin: Include/objimpl.h would > load Include/pycore/pycore_objimpl.h if Py_BUILD_CORE is defined: > > #ifdef Py_BUILD_CORE > # include "pycore/pycore_objimpl.h" > #endif I don't think more or less API should be magically included based on whether Py_BUILD_CORE is defined or not. If we want to have private headers, we should include them where needed and not install them. Really, Py_BUILD_CORE should go away. We should be moving away from monolithic includes like Python.h to having each C file include exactly what it uses, private or not. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rename Include/internals/ to Include/pycore/
Le dim. 28 oct. 2018 à 21:50, Benjamin Peterson a écrit : > I don't think more or less API should be magically included based on whether > Py_BUILD_CORE is defined or not. If we want to have private headers, we > should include them where needed and not install them. Really, Py_BUILD_CORE > should go away. We should be moving away from monolithic includes like > Python.h to having each C file include exactly what it uses, private or not. I would prefer to avoid annoying with the backward compatibility. Currently, #include more or less provides you "anything" and I'm fine with that. I prefer to no put too many changes at once :-) My overall approach is to make sure that we don't leak functions by mistakes into the public API or into the stable API anymore. For example, if a function is really for the core, put it in pycore/. It will be more explicit when reviewing a change for example. Py_BUILD_CORE is not only used to select which functions you get. Py_BUILD_CORE is also commonly used to get a macro instead of a function call, for best performances. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rename Include/internals/ to Include/pycore/
On Sun, Oct 28, 2018, at 14:30, Victor Stinner wrote: > Le dim. 28 oct. 2018 à 21:50, Benjamin Peterson a écrit > : > > I don't think more or less API should be magically included based on > > whether Py_BUILD_CORE is defined or not. If we want to have private > > headers, we should include them where needed and not install them. Really, > > Py_BUILD_CORE should go away. We should be moving away from monolithic > > includes like Python.h to having each C file include exactly what it uses, > > private or not. > > I would prefer to avoid annoying with the backward compatibility. > Currently, #include more or less provides you "anything" > and I'm fine with that. It doesn't break backward compatibility if you only make this required for new APIs. > > I prefer to no put too many changes at once :-) > > My overall approach is to make sure that we don't leak functions by > mistakes into the public API or into the stable API anymore. For > example, if a function is really for the core, put it in pycore/. It > will be more explicit when reviewing a change for example. How does the current Include/internal/ directory fail at accomplishing your goal? > > Py_BUILD_CORE is not only used to select which functions you get. > Py_BUILD_CORE is also commonly used to get a macro instead of a > function call, for best performances. I think the macro and function versions should just have different names then. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
