[Python-Dev] RM for 3.6?
Isn't it a time to assign release manager for 3.6-3.7? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On 30 May 2015 10:46, "Alexander Walters" wrote: > > Python is a giant cache-miss generator. A little performance boost on the > opt-code dispatch isn't going to change that much. If we really do care > about improving python to do less environmental damage, then that is a > discussion we should be having on it's own merits. It was really out of > place, even in this tangenty thread. I think the way core development gets funded is entirely on topic for the main core development mailing list, we just historically haven't discussed it openly, even though some of us have been advocating for various improvements to arrangements behind the scenes. I personally consider becoming more transparent about how we go about that process to be a good thing. Intel are looking to get involved in CPython core development *specifically* to work on performance improvements, so it's important to offer folks in the community good reasons for why we're OK with seeing at least some of that work applied to Python 2, rather than restricting their contributions to Python 3. The key is that the reason for not backporting performance enhancements *differs* from the reasons for not backporting new features. Rolling out new features has a broad ripple effect on the Python ecosystem as books, training material, etc, all need to be updated, and projects need to decide how to communicate their version dependencies appropriately if they decide to depend on one of the backported features. We pushed that kind of Python 2 change out to PyPI years ago, and aside from truly exceptional cases like the network security enhancements in PEPs 466 & 476 and the decision to bundle pip to make it easier to access PyPI, it isn't open for reconsideration as a general principle. Performance improvements, by contrast, historically haven't been backported solely due to the stability and maintainability implications for CPython itself - they don't have a change management ripple effect the way new language and standard library level features do. That lack of negative ripple effects that cause work for other people is why the proposal to contribute paid development time makes such a big difference to the acceptability of Python 2.7 performance patches, as it should be a pure gain for current Python 2.7 users, and the paid development contributions should address the maintainability concerns on the core development side (particularly since Intel are *paying* for their coaching in core contribution practices and politics, rather than expecting to receive that coaching from community volunteers for free). Backporting the computed goto patch is an easy place for them to start, since the change is already well tested in the Python 3 branch, but we don't expect this to be the end of the line for CPython 2 (or 3) performance enhancements. However, we also shouldn't downplay the significance of this as a notable policy change for the Python 2.7 maintenance branch, which means it is useful to offer folks as many reasons as we can to help them come to terms with the idea that Python 2 performance still matters, and that it is only the limitations on our development and support capacity that prevented us from further improving it previously. The commercially pragmatic reason is because Python 2 is where the largest current installed base is today, so applying some of the increased development capacity arising from sponsored contributions to Python 2.7 performance improvements is a good way to demonstrate to Python 2 developers that we still care about them *as Python 2 users*, rather than only being interested in them as potential future Python 3 users. This is the rationale that's likely to get our paid contributors (both current and future) on board with the idea, but it isn't necessarily going to be compelling to folks that are here as volunteers. The first "What's in it for the volunteers?" reason is the one I raised: giving the nod to an increased corporate developer presence in Python 2 maintenance should eventually let volunteers stop worrying about even Python 2.7 bug fix changes with a clear conscience, confident that as volunteer efforts drop away redistributors and other folks with an institutional interest will pick up the slack with paid development time. "Do the fun stuff for free, figure out a way to get paid for the boring-but-necessary stuff (or leave those tasks to someone else that's getting paid to handle them)" is a good sustainable approach to open source development, while trying to do it *all* for free is a fast path to burnout. Being ready, willing and able to handle the kind of situation created by the Python 2->3 community transition is a large part of what it means to offer commercial support for community driven open source projects, as it buys customers' time for either migration technologies to mature to a point where the cost of migration drops dramatically, for the newer version of a platform to move far enough ahead of
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On Sat, 30 May 2015 10:34:15 +1000 Nick Coghlan wrote: > On 30 May 2015 09:57, "Antoine Pitrou" wrote: > > > > On Sat, 30 May 2015 01:49:10 +0200 > > Christian Heimes wrote: > > > For performance patches we have to consider our responsibility for the > > > environment. Every improvement means more speed and less power > > > consumption. Python runs of hundreds of thousands of machines in the > > > cloud. Python 2.7 will be used for at least half a decade, probably > > > longer. Servers can be replaced with faster machines later and less > > > fossil fuel must be burned to produce power. > > > > Please keep your ideology out of this. > > I'm a qualified engineer (in computer systems engineering), so caring about > environmental sustainability is part of my professional ethical standards, > not just a matter of personal preference: http://www.wfeo.org/ethics/ There is no reason to assume that a smallish performance improvement in a single Python 2.7 release will make any difference in "environmental sustainability" of the world's computing infrastructure, while the problem is measured in orders of magnitude. The onus is on to you to prove the contrary. Otherwise, bringing it up is mere ideology. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On Sat, May 30, 2015 at 8:35 PM, Antoine Pitrou wrote: > On Sat, 30 May 2015 10:34:15 +1000 > Nick Coghlan wrote: >> On 30 May 2015 09:57, "Antoine Pitrou" wrote: >> > >> > On Sat, 30 May 2015 01:49:10 +0200 >> > Christian Heimes wrote: >> > > For performance patches we have to consider our responsibility for the >> > > environment. Every improvement means more speed and less power >> > > consumption. Python runs of hundreds of thousands of machines in the >> > > cloud. Python 2.7 will be used for at least half a decade, probably >> > > longer. Servers can be replaced with faster machines later and less >> > > fossil fuel must be burned to produce power. >> > >> > Please keep your ideology out of this. >> >> I'm a qualified engineer (in computer systems engineering), so caring about >> environmental sustainability is part of my professional ethical standards, >> not just a matter of personal preference: http://www.wfeo.org/ethics/ > > There is no reason to assume that a smallish performance improvement in > a single Python 2.7 release will make any difference in "environmental > sustainability" of the world's computing infrastructure, while the > problem is measured in orders of magnitude. The onus is on to you to > prove the contrary. Otherwise, bringing it up is mere ideology. The magnitude of the environmental benefit of Python performance improvement is uncertain, but we know what direction it's going to be. If there's going to be a massive maintenance nightmare, or if the change comes at a cost of functionality or debuggability, then sure, the onus is on the person begging for performance improvements; but if there's no such cost (or if the cost is being carried by the same person/people who proposed the change), then surely it's worth something? Suppose someone came up with a magic patch that makes the CPython core run 25% faster. No downsides, just 25% faster across the board. I wouldn't pay money for it on the sole basis of expecting to make that back in reduced electricity bills, but I certainly wouldn't be sorry to watch the load averages drop. Why is this controversial? ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On Sat, 30 May 2015 18:55:20 +1000 Nick Coghlan wrote: > On 30 May 2015 10:46, "Alexander Walters" wrote: > > > > Python is a giant cache-miss generator. A little performance boost on the > > opt-code dispatch isn't going to change that much. If we really do care > > about improving python to do less environmental damage, then that is a > > discussion we should be having on it's own merits. It was really out of > > place, even in this tangenty thread. > > I think the way core development gets funded is entirely on topic for > the main core development mailing list, we just historically haven't > discussed it openly, even though some of us have been advocating for > various improvements to arrangements behind the scenes. I personally > consider becoming more transparent about how we go about that process > to be a good thing. The way this so-called discussion is taking place feels much less like an actual discussion than an aggressive push for a change in maintenance policy. Guido has already taunted Ian Cordasco out of contributing. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On 30 May 2015 at 20:35, Antoine Pitrou wrote: > On Sat, 30 May 2015 10:34:15 +1000 > Nick Coghlan wrote: >> On 30 May 2015 09:57, "Antoine Pitrou" wrote: >> > >> > On Sat, 30 May 2015 01:49:10 +0200 >> > Christian Heimes wrote: >> > > For performance patches we have to consider our responsibility for the >> > > environment. Every improvement means more speed and less power >> > > consumption. Python runs of hundreds of thousands of machines in the >> > > cloud. Python 2.7 will be used for at least half a decade, probably >> > > longer. Servers can be replaced with faster machines later and less >> > > fossil fuel must be burned to produce power. >> > >> > Please keep your ideology out of this. >> >> I'm a qualified engineer (in computer systems engineering), so caring about >> environmental sustainability is part of my professional ethical standards, >> not just a matter of personal preference: http://www.wfeo.org/ethics/ > > There is no reason to assume that a smallish performance improvement in > a single Python 2.7 release will make any difference in "environmental > sustainability" of the world's computing infrastructure, while the > problem is measured in orders of magnitude. The onus is on to you to > prove the contrary. Otherwise, bringing it up is mere ideology. This isn't about this one change - it's about changing the Python 2.7 maintenance policy to allow ongoing performance improvements to Python 2.7, backed by additional commercial investment in Python 2.7 maintenance to mitigate the increased risks to stability and maintainability. As I say in my other email, though, not all of our volunteers are going to care about the fact that there are a lot of institutional downstream users of Python 2.7 that will appreciate this change in policy (e.g. all of the open government data sites running on CKAN: http://ckan.org/instances ), as well as the sponsored contributions that make it feasible. If the environmental benefits (however unquantifiable) help some folks to see the value in the change in policy, then that's a good thing, even if it's not the actual primary motivation for the change (the latter honor belongs to the fact that folks at Intel are interested in working on it, and they've backed that interest up both by joining the PSF as a sponsor member, and by hiring David Murray's firm to help coach them through the process). As strings go, "we want to work on improving Python 2.7 performance, not just Python 3 performance" isn't a bad one to have attached to a credible offer of ongoing contributions to CPython development :) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On Sat, 30 May 2015 20:52:21 +1000 Chris Angelico wrote: > > Suppose someone came up with a magic patch that makes the CPython core > run 25% faster. No downsides, just 25% faster across the board. I > wouldn't pay money for it on the sole basis of expecting to make that > back in reduced electricity bills, but I certainly wouldn't be sorry > to watch the load averages drop. Why is this controversial? That was not my point. What I'm opposing is the idea that "environmental sustainability" (or what people's ideological conception of it is) should become part of our criteria when making maintenance decisions. Obviously if a patch makes CPython faster without any downsides, there is no need to argue about environmental sustainability to make the patch desirable. The performance improvement itself is a sufficient reason. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On Sat, May 30, 2015 at 9:00 PM, Antoine Pitrou wrote: > On Sat, 30 May 2015 20:52:21 +1000 > Chris Angelico wrote: >> >> Suppose someone came up with a magic patch that makes the CPython core >> run 25% faster. No downsides, just 25% faster across the board. I >> wouldn't pay money for it on the sole basis of expecting to make that >> back in reduced electricity bills, but I certainly wouldn't be sorry >> to watch the load averages drop. Why is this controversial? > > That was not my point. What I'm opposing is the idea that > "environmental sustainability" (or what people's ideological conception > of it is) should become part of our criteria when making maintenance > decisions. > > Obviously if a patch makes CPython faster without any downsides, there > is no need to argue about environmental sustainability to make the > patch desirable. The performance improvement itself is a sufficient > reason. Okay. But what objection do you have to reduced electricity usage? I'm still not understanding how this is a problem. It might not be a priority for everyone, but surely it's a nice bonus? ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On 30 May 2015 at 20:58, Antoine Pitrou wrote: > On Sat, 30 May 2015 18:55:20 +1000 > Nick Coghlan wrote: >> On 30 May 2015 10:46, "Alexander Walters" wrote: >> > >> > Python is a giant cache-miss generator. A little performance boost on the >> > opt-code dispatch isn't going to change that much. If we really do care >> > about improving python to do less environmental damage, then that is a >> > discussion we should be having on it's own merits. It was really out of >> > place, even in this tangenty thread. >> >> I think the way core development gets funded is entirely on topic for >> the main core development mailing list, we just historically haven't >> discussed it openly, even though some of us have been advocating for >> various improvements to arrangements behind the scenes. I personally >> consider becoming more transparent about how we go about that process >> to be a good thing. > > The way this so-called discussion is taking place feels much less like > an actual discussion than an aggressive push for a change in maintenance > policy. Guido has already taunted Ian Cordasco out of contributing. Ian was unfortunately responding from incomplete information. While "we're all volunteers here" was true for a very long time, with Guido being the main exception since the PythonLabs days, a number of folks (both existing core contributors and members of other organisations) have been working hard to change that, since it's an unsustainable state of affairs given the criticality of CPython as a piece of Internet infrastructure. Given the extensive complaints about the lack of corporate contribution to upstream CPython maintenance, the hostile reaction to a concrete proposal for such ongoing contributions has been both incredibly surprising *and* disappointing, especially when it was deliberately aimed at tasks that most volunteers find to be a unrewarding chore rather than an entertaining use of their free time. The offer came with one string attached: that the Python 2.7 branch be opened up for performance improvements in addition to bug fixes. Since maintainability was the main concern with not backporting performance improvements in the first place, this seemed like a straight up win to me (and presumably to other folks aware of the offer), so it never even occurred to us that folks might not accept "because this proposal is backed by a credible offer of ongoing contributions to CPython maintenance and support" as a complete answer to the question of "Why accept this proposal to backport performance enhancements, and not previous proposals?". Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On Sat, 30 May 2015 21:20:56 +1000 Nick Coghlan wrote: > Given the extensive complaints about the lack of corporate > contribution to upstream CPython maintenance, the hostile reaction to > a concrete proposal for such ongoing contributions has been both > incredibly surprising *and* disappointing IMHO, they were not more hostile than against individuals' contributions of the same kind. Any patch proposal is bound to controversy, that's a normal aspect of the process, and one that contributors should usually be willing to go through. Also, when there are in rules in place, most people want to see them upholded, because that tends to promote fairness much more than when exceptions are granted all over the place. So people's reactions have really been understandable, if debatable. (FTR, Intel contacted me in private about such contributions and I said the backport of the computed gotos sounded ok to me -- since it has turned out entirely harmless on the 3.x branches --; that doesn't mean I like how this public discussion has turned out) > The offer came with one string attached: that the Python 2.7 branch be > opened up for performance improvements in addition to bug fixes. Since > maintainability was the main concern with not backporting performance > improvements in the first place, this seemed like a straight up win to > me (and presumably to other folks aware of the offer), so it never > even occurred to us > that folks might not accept "because this proposal > is backed by a credible offer of ongoing contributions to CPython > maintenance and support" as a complete answer to the question of "Why > accept this proposal to backport performance enhancements, and not > previous proposals?". You're making contribution some kind of contractual engagement. That's not an obvious improvement, because it has some large impacts on the power structure (for one, volunteers can't reasonably compete with contractual engagements). Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
Antoine Pitrou writes: > On Sat, 30 May 2015 01:49:10 +0200 > Christian Heimes wrote: > > For performance patches we have to consider our responsibility for the > > environment. Every improvement means more speed and less power > > consumption. Python runs of hundreds of thousands of machines in the > > cloud. Python 2.7 will be used for at least half a decade, probably > > longer. Servers can be replaced with faster machines later and less > > fossil fuel must be burned to produce power. > > Please keep your ideology out of this. Bad idea, unless you have benchmarks and engineering studies proving that that effect doesn't exist and never will. In a community of volunteers, ideology is typically a great motivator. If it weren't for ideology (specifically, RMS's), many of us wouldn't be here, and quite likely nothing like the current Linux and BSD ecosystems would be available yet, and maybe not at all. Which points to a better idea: Harness ideology to encourage contributions that help us all. Hey, Christian, maybe you know some sustainability advocates who'd like to help fund that work? Or do the programming? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
Le 30/05/2015 13:51, Stephen J. Turnbull a écrit : > Antoine Pitrou writes: > > On Sat, 30 May 2015 01:49:10 +0200 > > Christian Heimes wrote: > > > For performance patches we have to consider our responsibility for the > > > environment. Every improvement means more speed and less power > > > consumption. Python runs of hundreds of thousands of machines in the > > > cloud. Python 2.7 will be used for at least half a decade, probably > > > longer. Servers can be replaced with faster machines later and less > > > fossil fuel must be burned to produce power. > > > > Please keep your ideology out of this. > > Bad idea, unless you have benchmarks and engineering studies proving > that that effect doesn't exist and never will. No, it's up to the proponent to prove that the effect exists, with a magnitude large enough to make any interesting difference. That's part of the process when suggesting a change. If it doesn't, or if it's entirely cosmetical, it may be an important part of Christian's lifestyle (as are many individual practices, including religious, militant, dietetic...), but it certainly shouldn't brought up here. We don't want everyone trying to inject their beliefs in the maintenance process. > In a community of volunteers, ideology is typically a great motivator. If and only everyone agrees on it. Otherwise, it is typically a great divisor. Even abidance to RMS' writings and actions would probably not be unanimous here... Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 2015-05-30 14:03, Antoine Pitrou wrote: > No, it's up to the proponent to prove that the effect exists, with > a magnitude large enough to make any interesting difference. That's > part of the process when suggesting a change. > > If it doesn't, or if it's entirely cosmetical, it may be an > important part of Christian's lifestyle (as are many individual > practices, including religious, militant, dietetic...), but it > certainly shouldn't brought up here. We don't want everyone trying > to inject their beliefs in the maintenance process. Antoine, now your are putting it over the top. You make it sound like I'm some crazy environmentalist or eco-warrior. Well, I'm not. I merely keep the environment in mind. Yes, I have a modern, power saving washing machine and LED lights at home. Mostly because they save money in the long run (Germany's electricity prices are high). That was also the point behind my comment. Increased performance result in better efficiency which lead to better utilization of hardware and less power consumption. Companies are interested in better efficiency, because they have to pay less for hardware, power and cooling. The obvious benefits for our environment are a side effect. A smaller CO2 foot print is not my main concern. But I wanted to bring it up anyway. For some it is an small additional motivator for performance improvements. For others it could be a marketing instrument. In Germany ads are full of crazy 'green' slogans. Christian -BEGIN PGP SIGNATURE- iQEcBAEBCgAGBQJVabPTAAoJEIZoUkkhLbaJTF0H+wb8ciikP762qc8u586H2AjV 2xV9AAamI1Z6RwlvKRM7YHVk48coYIKk9WQ6DZODNlVSIhnijexII1dai91gbQvy jEVkLK2P6/C7I4gz7Fp0/SoCwkpGCev2CiSJUhIoE4oIw+Mm4BRASpf5hn4n+pRI yqXixYf7h+QWHgN0FRU3GU8RxNYRe65zB/3YeDUhKLQdkf8Gq4NVX7rlTx1gvZrq DbaGjKtkT8uec6hnvZcXwWVODYW10VHTonhlV3ff0sReXw94sXOeQwQ3n+7uwKAb sqvy11k0r6JejNGFxJqfMyXH557LP5ucc2g9+J8M2Sw4SOs7L6E+caaX89FY754= =soyL -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On 30 May 2015 at 21:37, Antoine Pitrou wrote: > On Sat, 30 May 2015 21:20:56 +1000 > Nick Coghlan wrote: >> Given the extensive complaints about the lack of corporate >> contribution to upstream CPython maintenance, the hostile reaction to >> a concrete proposal for such ongoing contributions has been both >> incredibly surprising *and* disappointing > > IMHO, they were not more hostile than against individuals' > contributions of the same kind. Any patch proposal is bound to > controversy, that's a normal aspect of the process, and one that > contributors should usually be willing to go through. > > Also, when there are in rules in place, most people want to see them > upholded, because that tends to promote fairness much more than when > exceptions are granted all over the place. So people's reactions have > really been understandable, if debatable. Agreed, but it's also understandable when folks forget that things that they're taking for granted aren't necessarily common knowledge. In this case: * the fact that this proposal was a suggested starting point for ongoing contributions, not a one-and-done effort * the fact that the rationale for the prohibition on performance enhancements was *different* from the reason for disallowing new features (and hence requiring a PEP for *any* new Python 2.7 feature) For folks that already knew both those facts, this *wasn't* a controversial suggestion. We unfortunately failed to account for the fact that not everyone was aware of that context, and that *is* a highly regrettable mistake. Hence my information dumps thoughout the thread, attempting to provide that context without committing folks to things they haven't committed to, and without disclosing potentially confidential third party information. >> The offer came with one string attached: that the Python 2.7 branch be >> opened up for performance improvements in addition to bug fixes. Since >> maintainability was the main concern with not backporting performance >> improvements in the first place, this seemed like a straight up win to >> me (and presumably to other folks aware of the offer), so it never >> even occurred to us >> that folks might not accept "because this proposal >> is backed by a credible offer of ongoing contributions to CPython >> maintenance and support" as a complete answer to the question of "Why >> accept this proposal to backport performance enhancements, and not >> previous proposals?". > > You're making contribution some kind of contractual engagement. That's > not an obvious improvement, because it has some large impacts on the > power structure (for one, volunteers can't reasonably compete with > contractual engagements). We've long had a requirement that certain kinds of proposal come with at least nominal support commitments from the folks proposing them (e.g. adding modules to the standard library, supporting new platforms). Institutions with a clear financial interest in a particular problem area can certainly make such commitments more credibly, so I agree with your concerns about the potential impact on the power dynamics of core development. That's one of the main benefits I see in attempting to guide sponsored contributions towards Python 2.7, at least initially - that's in LTS mode, so working on it is fairly uninteresting anyway, and it keeps discussion of *new* features (and hence the overall direction of language evolution) a community focused activity. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
Who said anything about funding? this is a thread about the patch Intel offered (and had committed). And that's the point. This is the thread about THAT patch. Why are we hijacking this topic for an environmental debate? If it is a legitimate topic (which it might be), discuss it in its own right. Otherwise it sounds like guilt-tripping and greenwashing. This patch will do little to nothing statistically significant for the environment. Bringing that up is ideology and politics. On 5/30/2015 04:55, Nick Coghlan wrote: On 30 May 2015 10:46, "Alexander Walters" wrote: Python is a giant cache-miss generator. A little performance boost on the opt-code dispatch isn't going to change that much. If we really do care about improving python to do less environmental damage, then that is a discussion we should be having on it's own merits. It was really out of place, even in this tangenty thread. I think the way core development gets funded is entirely on topic for the main core development mailing list, we just historically haven't discussed it openly, even though some of us have been advocating for various improvements to arrangements behind the scenes. I personally consider becoming more transparent about how we go about that process to be a good thing. Intel are looking to get involved in CPython core development *specifically* to work on performance improvements, so it's important to offer folks in the community good reasons for why we're OK with seeing at least some of that work applied to Python 2, rather than restricting their contributions to Python 3. The key is that the reason for not backporting performance enhancements *differs* from the reasons for not backporting new features. Rolling out new features has a broad ripple effect on the Python ecosystem as books, training material, etc, all need to be updated, and projects need to decide how to communicate their version dependencies appropriately if they decide to depend on one of the backported features. We pushed that kind of Python 2 change out to PyPI years ago, and aside from truly exceptional cases like the network security enhancements in PEPs 466 & 476 and the decision to bundle pip to make it easier to access PyPI, it isn't open for reconsideration as a general principle. Performance improvements, by contrast, historically haven't been backported solely due to the stability and maintainability implications for CPython itself - they don't have a change management ripple effect the way new language and standard library level features do. That lack of negative ripple effects that cause work for other people is why the proposal to contribute paid development time makes such a big difference to the acceptability of Python 2.7 performance patches, as it should be a pure gain for current Python 2.7 users, and the paid development contributions should address the maintainability concerns on the core development side (particularly since Intel are *paying* for their coaching in core contribution practices and politics, rather than expecting to receive that coaching from community volunteers for free). Backporting the computed goto patch is an easy place for them to start, since the change is already well tested in the Python 3 branch, but we don't expect this to be the end of the line for CPython 2 (or 3) performance enhancements. However, we also shouldn't downplay the significance of this as a notable policy change for the Python 2.7 maintenance branch, which means it is useful to offer folks as many reasons as we can to help them come to terms with the idea that Python 2 performance still matters, and that it is only the limitations on our development and support capacity that prevented us from further improving it previously. The commercially pragmatic reason is because Python 2 is where the largest current installed base is today, so applying some of the increased development capacity arising from sponsored contributions to Python 2.7 performance improvements is a good way to demonstrate to Python 2 developers that we still care about them *as Python 2 users*, rather than only being interested in them as potential future Python 3 users. This is the rationale that's likely to get our paid contributors (both current and future) on board with the idea, but it isn't necessarily going to be compelling to folks that are here as volunteers. The first "What's in it for the volunteers?" reason is the one I raised: giving the nod to an increased corporate developer presence in Python 2 maintenance should eventually let volunteers stop worrying about even Python 2.7 bug fix changes with a clear conscience, confident that as volunteer efforts drop away redistributors and other folks with an institutional interest will pick up the slack with paid development time. "Do the fun stuff for free, figure out a way to get paid for the boring-but-necessary stuff (or leave those tasks to someone else that's getting paid to handle them)" is
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
Antoine Pitrou writes: > > In a community of volunteers, ideology is typically a great > > motivator. > > If and only everyone agrees on it. That, my friend, is *your* ideology speaking. Some people work on open source to scratch technical itches -- the program doesn't do what they want, they're able to improve it, the license allows them to improve it, so they do, done. Others use that same freedom to change the software to improve the world in other ways. We don't need to agree on *why* we do the work we do. We only need to agree on an evaluation and arbitration process for determining *which* work gets released as part of "original Python". More on that below. > Otherwise, it is typically a great divisor. Only because some people make a point of insisting on implementing theirs[1] -- and others insist on objecting to any mention of it. I think both extremes are divisive -- but nothing new there, extremes usually are divisive. Now, Christian did say "must" when he suggested considering the environment, and that's obviously not right. To the extent that folks are volunteers and not bound by the professional ethics that Nick professes, there's no *must* about it. I don't think Christian really meant to try to impose that on everybody in the project, though. It was more a wish on his part as I understand it, one he knows will at best be fulfilled gradually and voluntarily as people come to be aware of the issue and agree with him that some things need to be done to address it. But if people like Christian choose to work on patches because they are "environmentally friendly", or vote +1 on them, even if that means a clarification or even reinterpretation of maintenance policy, why should we care whether they say what their motivation is? On the other hand, if it *is* a change in maintenance policy to commit the Intel patch, IMO you have right on your side to speak up about that (as you do elsewhere). (OTOH, it seems to me that most posters in this thread so far agree that it's a mere clarification of *policy*, although it's a clear reallocation of *effort* that probably wouldn't come voluntarily from the core committers.) You're also right to point out that the nature of the community will change as people paid to work on commercially desirable tasks become committers. Definitely the natures of Linux and GUI framework development changed (as indeed X11 did when it passed from a commercial consortium to a more open organization) as commercial interests started supplying more and more effort, as well as hiring core developers. Whether that prospective change is a good thing for Python is a matter for debate, and (speaking only for myself, and this may not be the appropriate channel anyway) I'm interested in hearing your discussion on that matter. > Even abidance to RMS' writings and actions would probably not > be unanimous here... I assure there's absolutely no "probably" about it. You evidently missed the (intended though obscure) irony of *me* praising RMS's ideology (see return address). Footnotes: [1] You could argue that "insisting on implementing" is implied by "ideology", but then I expect that Christian would deny a desire to *impose* his values on the project. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
Hi Christian, > Antoine, > > now your are putting it over the top. You make it sound like I'm some > crazy environmentalist or eco-warrior. Well, I'm not. I apologize for misrepresenting your position. I still don't think discussing environmental matters is really productive here, though :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On May 30, 2015 1:56 AM, "Nick Coghlan" wrote: > > Being ready, willing and able to handle the kind of situation created > by the Python 2->3 community transition is a large part of what it > means to offer commercial support for community driven open source > projects, as it buys customers' time for either migration technologies > to mature to a point where the cost of migration drops dramatically, > for the newer version of a platform to move far enough ahead of the > legacy version for there to be a clear and compelling business case > for forward porting existing software, or (as is the case we're aiming > to engineer for Python), both. > Earlier, you said that it had been a surprise that people were against this change. I'd just point out that the reason is bound up in what you say here. Porting performance features from python 3 to python 2 has the disadvantage of cutting into a compelling business case for users to move forward to python 3.[1] so doing this has a cost to python 3 adoption. But, the question is whether there is a benefit that outweighs that cost. I think seeing more steady, reliable contributors to python core is a very large payment. Sure, for now that payment is aimed at extending the legs on the legacy version of python but at some point in the future python 2's legs will be well and truly exhausted. When that happens both the developers who have gained the skill of contributing to cpython and the companies who have invested money in training people to be cpython contributors will have to decide whether to give up on all of that or continue to utilize those skills and investments by bettering python 3. I'd hope that we can prove ourselves a welcoming enough community that they'd choose to stay. -Toshio [1] In fact, performance differences are a rather safe way to build compelling business cases for forwards porting. Safe because it is a difference (unlike api and feature differences) that will not negatively affect your ability to incrementally move your code to python 3. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On May 30, 2015, at 06:55 PM, Nick Coghlan wrote: >Intel are looking to get involved in CPython core development >*specifically* to work on performance improvements, so it's important >to offer folks in the community good reasons for why we're OK with >seeing at least some of that work applied to Python 2, rather than >restricting their contributions to Python 3. I think that's fine, for all the reasons you, Toshio, and others mention. For better or worse, Python 2.7 *is* our LTS release so I think we can make life easier for the folks stuck on it . However, I want us to be very careful not to accept performance improvements in Python 2.7 that haven't also been applied to Python 3, unless of course they aren't relevant. Python 3 also has a need for performance improvements, perhaps more so for various reasons, so let's make sure we're pushing that forward too. In many cases where you have a long lived stable release and active development releases, it's generally the policy that fixes show up in the dev release first. At least, this is the case with Ubuntu and SRUs, and it makes a lot of sense. Cheers, -Barry ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
For now, I'm following the mailing-lists from a spy-glass: I don't read most of the e-mails. However, this thread seems to be "infected": I can smell from here your emotions behind your words. Why to push a lot of emotions inside a technical discussion ? What's the nerves have been hit with this discussion ? If you know me a little bit, you know I'm always interested in by efficiency improvements, especially around Python. However, I see two parts of this discussion: 1. Python 3 must continue to be the first class citizen for the features, bugs-killing and performance improvements, as Barry explained. Programming in Python isn't only a language, it's also a spirit and a community with forces and weaknesses. The main "issue" for the Python 3 adoption by the community is that Python community is mainly composed by Late Majority and Laggards [1], contrary to some fancy programming language like Ruby, Go, Rust, where you have a majority of Early Adopters. For example, the migration from Ruby 1.8 to 1.9 has taken time because they changed some critical parts, but finally, now, almost nobody uses Ruby 1.8 on production. FYI, Ruby 1.9 has been released only one year after Python 3.0, and Ruby community has finished their migration a long time ago, where you continue to support Python 2.7. Maybe the change was less important between Ruby 1.8 and 1.9 that between Python 2 and Python 3, however I personally think the majority of Early Adopters in Ruby community has helped a lot for that. Nevertheless, at least to my eyes, it's a proof that, despite the fact time to time somebody announce that Python is dying and that nobody will use that on production for the new projects, in fact, Python is a clearly a mainstream programming language, Python 3 migration time is the best proof, you don't have that with the fancy languages. But, it also means that to accelerate Python 3 adoption, we need more incentives: Have a clean way to migrate, almost important libraries ported and the fact that Python 3 is more newcomers friendly [2] aren't enough, new features and performances are a better incentive, at least to me. Without AsyncIO, I'll continue to code for Python 2. 2. From a strategical point of view, even if it should be reduce the adoption speed of Python 3, it should be a good "move" to support that for Python 2, to reduce the risk of fork of Python: It's better for the Python community to use Python 2 than not Python at all. See the NodeJS community: even if the reasons seem to be more political than technical, fork a language isn't a potential myth. If we force too much Python 2 users to migrate to Python 3, they should reject completely the language, everybody will lose in this story. Moreover, if we start to have a critical mass of Laggards with Python 2 who have enough money/time to maintain a patch like that, and we reject that, we should lose the discussion link and mutual enrichment: everybody is concerned by performance improvements. Personally, only final results matter, I don't care about the personal motivations: economical, ecological, or basely to publish a blog post about the fact that the Python community has a bigger one that some others ;-) And don't forget: Almost nobody cares about our internal discussions and our drama, they only interested by the source code we produce, even the Python developers who use CPython. Even if we have different motivations, I'm sure that everybody on this mailing-list, or at least in this thread, "believe" in Python: You don't take personal time during a week-end if Python isn't something important to you, because during the time you take to write e-mails/source code, you don't watch series or take care of your family. [1] http://en.wikipedia.org/wiki/Early_adopter#History [2] It's in French (Google translate is your friend), however an interesting point of view of a Python trainer who has switched to Python 3: http://sametmax.com/python-3-est-fait-pour-les-nouveaux-venus/ (The website is down for now) -- Ludovic Gasc (GMLudo) http://www.gmludo.eu/ 2015-05-30 17:42 GMT+02:00 Barry Warsaw : > On May 30, 2015, at 06:55 PM, Nick Coghlan wrote: > > >Intel are looking to get involved in CPython core development > >*specifically* to work on performance improvements, so it's important > >to offer folks in the community good reasons for why we're OK with > >seeing at least some of that work applied to Python 2, rather than > >restricting their contributions to Python 3. > > I think that's fine, for all the reasons you, Toshio, and others mention. > For > better or worse, Python 2.7 *is* our LTS release so I think we can make > life > easier for the folks stuck on it . > > However, I want us to be very careful not to accept performance > improvements > in Python 2.7 that haven't also been applied to Python 3, unless of course > they aren't relevant. Python 3 also has a need for performance > improvements, > perhaps more so for various reasons, so let's make sure we're
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On 31 May 2015 04:20, "Ludovic Gasc" wrote: > > For now, I'm following the mailing-lists from a spy-glass: I don't read most of the e-mails. > However, this thread seems to be "infected": I can smell from here your emotions behind your words. > > Why to push a lot of emotions inside a technical discussion ? > What's the nerves have been hit with this discussion ? I think you answered your own question fairly well - there's a longstanding, but rarely articulated, culture clash between the folks that are primarily interested in the innovators and early adopters side of things, and those of us that are most interested in bridging the gap to the early majority, late majority and laggards. Add in the perfectly reasonable wariness a lot of folks have regarding the potential for commercial interests to unfairly exploit open source contributors without an adequate return contribution of development effort, gratis software, gratis services, or interesting employment opportunities, and you're going to see the occasional flare-ups as we find those rough edges where differences in motivation & background lead to differences of opinion & behaviour. Cheers, Nick. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On 05/30/2015 07:26 AM, Toshio Kuratomi wrote: Porting performance features from python 3 to python 2 has the disadvantage of cutting into a compelling business case for users to move forward to python 3.[1] so doing this has a cost to python 3 adoption. But, the question is whether there is a benefit that outweighs that cost. [...] Backporting performance enhancements from 3 to 2 does seem to be counterproductive from the perspective of the Core Dev community. But certainly in this case, when Intel drops a major bundle of working code in our collective lap, it absolutely feels like the right thing to me to check it in and support it. And happily the Python Core Dev community generally does the right thing. Consider the flip side--what if we'd refused to accept it? What sort of signal would that be to the Python community? I don't know, but I'd guess that people would harbor ill will and distrust. I'd rather the community liked and trusted us; that makes it more likely they'll listen when we say "honest, Python 3 is better than 2--c'mon over!" //arry/ p.s. Supporting this patch also helps cut into PyPy's reported performance lead--that is, if they ever upgrade speed.pypy.org from comparing against Python *2.7.2*. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can someone configure the buildbots to build the 3.5 branch?
On Thu, May 28, 2015 at 6:59 PM, Larry Hastings wrote: > The buildbots currently live in a state of denial about the 3.5 branch. > Could someone whisper tenderly in their collective shell-like ears so that > they start building 3.5, in addition to 3.4 and trunk? The 3.5 branch seems to be set up on the buildbots, we'll see how it goes when somebody commits something to 3.5. -- Zach ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
Nick Coghlan wrote: We've long had a requirement that certain kinds of proposal come with at least nominal support commitments from the folks proposing them (e.g. adding modules to the standard library, supporting new platforms). Institutions with a clear financial interest in a particular problem area can certainly make such commitments more credibly, Are such commitments from commercial entities really any more reliable in the long term than anyone else's? Such entities can be expected to drop them as soon as they perceive them as no longer being in their financial interests. -- Greg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On 31 May 2015 at 09:20, Larry Hastings wrote: > On 05/30/2015 07:26 AM, Toshio Kuratomi wrote: > > Porting performance features from python 3 to python 2 has the disadvantage > of cutting into a compelling business case for users to move forward to > python 3.[1] so doing this has a cost to python 3 adoption. But, the > question is whether there is a benefit that outweighs that cost. [...] > > Backporting performance enhancements from 3 to 2 does seem to be > counterproductive from the perspective of the Core Dev community. But > certainly in this case, when Intel drops a major bundle of working code in > our collective lap, it absolutely feels like the right thing to me to check > it in and support it. And happily the Python Core Dev community generally > does the right thing. There's another benefit that I didn't think to mention earlier, which is that getting folks from Python 2 -> Python 3 isn't actually my major version adoption concern at the moment: I'm more interested in how I can persuade them to stop using Python *2.6*, which is still a higher proportion of PyPI downloads with an identifiable client version than Python 3 [1], and the relative proportions between them are likely to be even worse once we start venturing inside corporate firewalls where direct downloads from PyPI aren't permitted. While I suspect barriers to migration at the distro level carry a fair bit of the blame there (and we're working on those), performance improvements in the 2.7 branch help provide an additional carrot to assist in that process, complementing the stick of trying to educate the community at large that it's unrealistic and exploitative [2] for folks to expect free community support for versions of Python that are so old that not even the core development team support them any more (i.e. Python 2.6 and earlier). My one consolation is that the Python community are far from alone in struggling to win that fight against institutional inertia once folks have widely adopted a version of a product that "works for them". My theory is that folks will pay to be able to keep using these older systems because our industry doesn't have very good tools for quantifying the cost of the technical debt incurred by attempting to maintain the status quo in the face of an evolving ecosystem. As infrastructure change management practices improve (e.g. through ideas like Holistic Software's hybrid dynamic management [3]), and not only the platform level tools but also the related business models evolve to better support those approaches, I'm hoping we'll see things change for the better not just in terms of Python in particular, but in terms of institutional infrastructure as a whole. Cheers, Nick. [1] https://caremad.io/2015/04/a-year-of-pypi-downloads/ [2] http://www.curiousefficiency.org/posts/2015/04/stop-supporting-python26.html [3] http://www.holistic-software.com/hybrid-dynamic-model -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 is here until 2020, please don't call it a waste.
On 31 May 2015 at 08:37, Greg Ewing wrote: > Nick Coghlan wrote: > >> We've long had a requirement that certain kinds of proposal come with >> at least nominal support commitments from the folks proposing them >> (e.g. adding modules to the standard library, supporting new >> platforms). Institutions with a clear financial interest in a >> particular problem area can certainly make such commitments more >> credibly, > > Are such commitments from commercial entities really > any more reliable in the long term than anyone else's? > Such entities can be expected to drop them as soon as > they perceive them as no longer being in their financial > interests. Yes, if the credibility stems from the market situation and the financial incentives leading an organisation to make the offer, rather than from the personal interest of one or two key folks at that organisation. Structural incentives are harder to shift than personal interests, so this is a case where institutional inertia actually works for the community rather than against us. It's not an ironclad guarantee (since businesses fail, divisions get shut down, companies decide to exit markets, etc), but if we understand the business case backing an investment decision (whether that investment is in the form of funding, developer time, or both), that's genuinely more reliable than commitments from individuals (since we don't have the kind of ability to manage and distribute risk that larger organisations do). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
