[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
This is too-little, too-late (I was offline this past week), but for those who enjoy digging through historical archives, the Tcl folks had an interesting stubs mechanism that was *supposed* to solve the versioning issue (although I suspect it hasn’t actually done much in that regard) in addition to the “cross-platform dynamic linking is a royal pain” (circa 1999) issue: https://wiki.tcl-lang.org/page/Stubs Basically, you just use a vtable of function pointers exported by version, e.g. Tcl_8_0_Stubs, Tcl_8_1_Stubs, etc. The clients call each function via a layer of indirection, Tcl_8_0_Stubs->Tcl_Tell(), Tcl_8_1_Stubs->Tcl_Tell(), etc.; this is hidden by C macros. (From what I recall of COM, this is a rudimentary implementation of a COM interface.) I’m not sure I would recommend this today -- we were solving different problems back then (CAD workstations from various Unix vendors patched Frankenstein-esque library mismatches), and compiler optimizations weren’t as sophisticated. It had the nice advantage of crashing immediately due to a version mismatch rather than silently corrupting data structures that had undergone changes, etc. Back then, GNU Binutils symbol versioning was just adding to our headaches (again, telling our customers to fix their workstations wasn’t a plausible solution, and it was only on Linux, which was then a very small part of the installed base). > On Oct 1, 2019, at 1:45 AM, Victor Stinner wrote: > > Sorry for the bad timing. I dislike working under pressure. The > discussion on python-dev, the bug tracker and pull requests was really > interesting. > > The issue has been fixed: the whole idea of stable ABI for PyConfig > has been abandoned. If you want to switch to a different version of > Python when you embed Python, you must recompile your code. It was > always like that and the PEP 587 doesn't change anything. > > PyConfig_InitPythonConfig() and PyConfig_InitIsolatedConfig() > initialize PyConfig to sane default configuration. So if we add new > fields in Python 3.9, you don't have to manually initialize these > fields, except if you explicitly want to use these new fields to > benefit of new configuration options. > > Victor > > Le mar. 1 oct. 2019 à 00:38, Nick Coghlan a écrit : >> >> >> >> On Tue., 1 Oct. 2019, 6:47 am Victor Stinner, wrote: >>> >>> Hi back, >>> >>> It seems like "config.struct_size = sizeof(PyConfig);" syntax is "not >>> really liked". >>> >>> I don't see any traction to provide a stable ABI for embedded Python. >>> The consensus is more towards: "it doesn't work and we don't want to >>> bother with false promises" (nor add code for that). >>> >>> Since Lukasz is supposed to tag 3.8.0rc1 today, I propose to remove >>> anything related to "stable API" or "stable ABI" from the PEP 587 >>> right now. I wrote two PRs to remove struct_size: >>> >>> * CPython: https://github.com/python/cpython/pull/16500 >>> * PEP 587: https://github.com/python/peps/pull/1185 >>> >>> Later, if we decide to start proving a stable ABI for embedded Python, >>> we can still add a "version" or "struct_size" field to PyConfig later >>> (for example in Python 3.9). >> >> >> Thanks Victor, I think this is the right way for us to go, given the >> relative timing in the release cycle. >> >> The idea of some day expanding the stable ABI to cover embedding >> applications, not just extension modules, is an intriguing one, but I'm >> genuinely unsure anyone would ever actually use it. >> >> Cheers, >> Nick. >> >> >> >>> >>> Victor >>> ___ >>> Python-Dev mailing list -- [email protected] >>> To unsubscribe send an email to [email protected] >>> https://mail.python.org/mailman3/lists/python-dev.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/[email protected]/message/2JAJQA5OANFPXAJ3327RRPHPQLKVP2EW/ > > > > -- > Night gathers, and now my watch begins. It shall not end until my death. > ___ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/36RTEPNE6HSPMXGDGSHWDJN56GZRX4YU/ ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/U63GT4FYZN7PBG22ZJD522EPYKM3OPQP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.
Oddly, I did not get Mark's original e-mail, but am seeing replies here.
Piggybacking off of James' email here...
On 03/12/2019 16:15, Mark Shannon wrote:
> Hi Everyone,
>
> I am proposing a new PEP, still in draft form, to impose a limit of one
> million on various aspects of Python programs, such as the lines of code
> per module.
My main concern about this PEP is it doesn't specify the behavior when a given
limit is exceeded. Whether you choose 10 lines or 10 billion lines as the rule,
someone annoying (like me) is going to want to know what's going to happen if I
break the rule.
Non-exhaustively, you could:
1. Say the behavior is implementation defined
2. Physically prohibit the limit from being exceeded (limited by
construction/physics)
3. Generate a warning
4. Raise an exception early (during parse/analysis/bytecode generation)
5. Raise an exception during runtime
The first two will keep people who hate limits happy, but essentially give the
limit no teeth. The last three are meaningful but will upset people when a
previously valid program breaks.
1. The C and C++ standards are littered with limits (many of which you have to
violate to create a real-world program) that ultimately specify that the
resulting behavior is "implementation defined." Most general-purpose compilers
have reasonable implementations (e.g. I can actually end my file without a
newline and not have it call abort() or execve("/usr/bin/nethack"), behaviors
both allowed by the C99 standard). You could go this route, but the end result
isn't much better than not having done the PEP in the first place (beyond
having an Ivory Tower to sit upon and taunt the unwashed masses, "I told you
so," when you do decide to break their code).
Don't go this route unless absolutely necessary. Of course, the C/C++ standard
isn't for an implementation; this PEP has the luxury of addressing a single
implementation (CPython).
2. Many of Java's limits are by construction. You can't exceed 2**16 bytecode
instructions for a method because they only allocated a uint16_t (u2 in the
classfile spec) for the program counter in various places. (Bizarrely, the size
of the method itself is stored as a uint32_t/u4.) I believe these limits are
less useful because you'll never hit them in a running program; you simply
can't create an invalid program. This would be like saying the size of Python
bytecode is limited to the number of particles in the universe (~10**80). You
don't have to specify the consequences because physics won't let you violate
them.
This is more useful for documenting format limits, but probably doesn't achieve
what you're trying to achieve.
3. Realistically, this is probably what you'd have to do in the first version
for PEP adoption to get non-readers of python-dev@ ready, but, again, it
doesn't achieve what you're setting out to do. We'd still accept programs that
exceed these limits, and whatever optimizations that depend on these limits
being in place wouldn't work.
Which brings us to the real meat, 4&5.
Some limits don't really distinguish between these cases. Exceeding the total
bytecode size for a module, for example, would have to fail at bytecode
generation time (ignoring truly irrational behavior like silently truncating
the bytecode). But others aren't so cut-and-dry. For example, a module that is
compliant except for a single function that contains too many local variables.
Whether you do 4 or 5 isn't so obvious:
Pros of choosing 4 (exception at load):
* I'm alerted of errors early, before I start a 90-hour compute job, only to
have it crash in the write_output() function.
* Don't have to keep a poisoned function that your optimizers have to special
case.
Pros of choosing 5 (exception at runtime):
* If I never call that function (maybe it's something in a library I don't
use), I don't get penalized.
* In line with other Python (mis-)behaviors, e.g. raising NameError() at
runtime if you typo a variable name.
On Tue 12/03/19, 10:05 AM, "Rhodri James" wrote:
On 03/12/2019 16:15, Mark Shannon wrote:
> Isn't this "640K ought to be enough for anybody" again?
> ---
>
> The infamous 640K memory limit was a limit on machine usable resources.
> The proposed one million limit is a limit on human generated code.
>
> While it is possible that generated code could exceed the limit,
> it is easy for a code generator to modify its output to conform.
> The author has hit the 64K limit in the JVM on at least two occasions
> when generating Java code.
> The workarounds were relatively straightforward and
> probably wouldn't have been necessary with a limit of one million
> bytecodes or lines of code.
I can absolutely guarantee that this will come back and bite you.
Someone out there will be doing something more complicated than you
think is
[Python-Dev] Re: Should set objects maintain insertion order too?
On Mon 12/16/19, 9:59 AM, "David Mertz" mailto:[email protected]>> wrote: Admittedly, I was only lukewarm about making an insertion-order guarantee for dictionaries too. But for sets I feel more strongly opposed. Although it seems unlikely now, if some improved implementation of sets had the accidental side effects of making them ordered, I would still not want that to become a semantic guarantee. Eek… No accidental side effects whenever possible, please. People come to rely upon them (like that chemistry paper example[*]), and changing the assumptions results in a lot of breakage down the line. Changing the length of AWS identifiers (e.g. instances from i-1234abcd to i-0123456789abcdef) was a huge deal; even though the identifier length was never guaranteed, plenty of folks had created database schemata with VARCHAR(10) for instance ids, for example. Break assumptions from day 1. If some platforms happen to return sorted results, sort the other platforms or toss in a sorted(key=lambda el: random.randomint()) call on the sorting platform. If you’re creating custom identifiers, allocate twice as many bits as you think you’ll need to store it. Yes, this is bad user code, but I’m all for breaking bad user code in obvious ways sooner rather than subtle ways later, especially in a language like Python. [*] https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/ ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/TKVGVH5GHOL3YH7W55MEU2PHASSPY74M/ Code of Conduct: http://python.org/psf/codeofconduct/
