Re: [Python-Dev] PEP 385: the eol-type issue
>I haven't commented on this issue before because I can't really be > helpful. I just don't understand why hg is being considered before > it's Windows support is roughly equivalent to svn and cvs. Is it really that you don't *understand*? It's fairly easy: there was a PEP which offered a number of options, and there was BDFL pronouncement. This (BDFL pronouncement) is how Python has always worked, and, as a principle, it is a good and useful process. Now, the specific outcome of the process means that more work needs to be done. So we have a *second* PEP, and we have a lack of volunteers that help implementing it. The second PEP hasn't been approved yet (as it isn't complete, yet), so migration to hg is stalled. The primary volunteer (Dirkjan) has indicated that he can't help with that specific issue, so other volunteers need to step forward, or we cannot move to hg. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 5/08/2009 4:50 PM, Ben Finney wrote: Mark Hammond writes: On 5/08/2009 3:56 PM, Ben Finney wrote: Mark Hammond writes: Let's say I make a branch of the hg repo, myself and a few others work on it committing as we go, then attempt to merge back upstream. Let's say some of the early commits on that clone introduced "bad" line endings. […] The problem is the sequence of events happened in the first place. An extra burden is placed on the developer that will quickly get tiresome. I wouldn't personally be happy if that workflow became the norm. Ah, okay. In that case, the ultimate “problem” is that OS vendors entrenched their incompatible line-ending conventions instead of choosing a single standard. Any line-ending burden borne by developers is a result of that. Yeah - this happened around 1964 if wikipedia is any guide. If things were different, they'd be different. However, we live with the legacy of that stupid set of decisions and have no real option to resolve it permanently short of deprecating entire vistas of tools (or even entire operating systems). Agreed - so let's not solve it permanently. ... It's not a simple thing to solve, and many clever people have tried over the decades. As already mentioned in this thread, a capability similar to what svn or cvs offers would be sufficient. While a DVCS does offer unique challenges, it seems to me that doing something at commit time without requiring magic hooks be configured would go a long way to addressing the problem. Magic hooks on the official repo would then be considered the final fallback defense, but should rarely be invoked. At some point, the decision about how to handle line endings in cross-platform data needs to be punted to a human for a context-sensitive assessment, since (as can be seen) the above list of requirements is internally inconsistent and can't be relegated to a one-size-fits-all algorithm. I'm not sure what point you are trying to make, but I believe it *is* possible for a solution to be found here which will keep Windows users happy. I'm guessing you haven't had much practical experience with this problem, so probably don't see this is clearly as Windows users do. Cheers, Mark. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
> If things were different, they'd be different. However, we live with the > legacy of that stupid set of decisions and have no real option to > resolve it permanently short of deprecating entire vistas of tools (or > even entire operating systems). I think you missed the solution to the problem that Mark proposed (IIUC): a local commit to a hg repository should already get the line endings right, by automatically converting the file-to-be-committed into the repository line endings. This is what CVS has supported for more than ten years, and what svn supports for close-to ten years. > * distributed VCS has the job of preserving data as present on the > filesystem, including whatever line-ending convention is present in a > file. No, that's not true. Distributed VCS has the job to help the developer. That may mean to preserve the file as-is, or it may mean to convert the file on checkout and checkin. Which of these would be needed depends on the file, of course. > It's not a simple thing to solve, and many clever people have tried over > the decades. The fact that a centralised VCS can put the problem aside > by requiring an explicit, single decision in the repository, is no help > when addressing the constraints of a distributed VCS. Why do you say that? It's not true. The approach that has worked for the central repository can work just as well for a distributed repository. > At some point, the decision about how to handle line endings in > cross-platform data needs to be punted to a human for a > context-sensitive assessment, since (as can be seen) the above list of > requirements is internally inconsistent and can't be relegated to a > one-size-fits-all algorithm. Right - there needs to be a way for the user to specify what line endings to use. That's why both CVS and subversion have supported such configuration, on a per file basis, for many years. I can't see why hg couldn't, in principle, support the same configuration. Being a DVCS, such configuration would have to be part of the clone, of course, being versioned, and all that. I think hg is well capable of keeping versioned configuration information in the clone, as demonstrated by the .hgignore files. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 5/08/2009 5:35 PM, "Martin v. Löwis" wrote: Now, the specific outcome of the process means that more work needs to be done. So we have a *second* PEP, and we have a lack of volunteers that help implementing it. The second PEP hasn't been approved yet (as it isn't complete, yet), so migration to hg is stalled. The primary volunteer (Dirkjan) has indicated that he can't help with that specific issue, so other volunteers need to step forward, or we cannot move to hg. I don't recall Dirkjan saying he can't help with that issue - was it a lack of time, or a lack of understanding the problem/lack of a Windows environment? The problem I see is a lack of agreement about exactly what the solution entails. I believe there is general agreement win32text needs to be enhanced to support versioned 'rules'. But even with that, the only option I see is a truly cross-platform extension to implement these rules which every Python committer, regardless of operating-system, is expected to use - but that doesn't seem the consensus. As mentioned, I'm willing to lend manpower for this once there is agreement on something workable... Cheers, Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Mercurial migration: help needed
This is a repost from a month ago. It didn't get much feedback last time. I have now two items. In this thread, I'd like to collect things that ought to be done but where Dirkjan has indicated that he would prefer if somebody else did it. Item 1 -- The first item is build identification. If you want to work on this, please either provide a patch (for trunk and/or py3k), or (if you are a committer) create a subversion branch. It seems that Barry and I agree that for the maintenance branches, sys.subversion should be frozen, so we need actually two sets of patches: one that removes sys.subversion entirely, and the other that freezes the branch to the respective one, and freezes the subversion revision to None. Of course, it seems that the actual representation of branches hasn't been determined yet, so the build process integration may need to be changed if named branches aren't going to be used in the end. Anybody working on this should have good knowledge of the Python source code, Mercurial, and either autoconf or Visual Studio (preferably both). Item 2 -- The second item is line conversion hooks. Dj Gilcrease has posted a solution which he considers a hack himself. Mark Hammond has also volunteered, but it seems some volunteer needs to be "in charge", keeping track of a proposed solution until everybody agrees that it is a good solution. It may be that two solutions are necessary: a short-term one, that operates as a hook and has limitations, and a long-term one, that improves the hook system of Mercurial to implement the proper functionality (which then might get shipped with Mercurial in a cross-platform manner). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Mark Hammond writes: > As already mentioned in this thread, a capability similar to what svn > or cvs offers would be sufficient. That capability presented by centralised VCSen is entirely dependent on the fact that they *are* centralised. Using a distributed VCS means the same capability doesn't apply. > While a DVCS does offer unique challenges, it seems to me that doing > something at commit time without requiring magic hooks be configured > would go a long way to addressing the problem. The hand-waving “doing something” is exactly what needs to be solved. > Magic hooks on the official repo would then be considered the final > fallback defense, but should rarely be invoked. Right, so that's “capability similar to centralised VCS” out of consideration; I'm glad we agree in the end. > I'm not sure what point you are trying to make That I disagree with your position. You seem to think that the problem has an obvious solution, which is not true; and that choice of a distributed VCS should be delayed until the problem is solved, which I don't agree with. > but I believe it *is* possible for a solution to be found here which > will keep Windows users happy. I'm guessing you haven't had much > practical experience with this problem, so probably don't see this is > clearly as Windows users do. Your guess is incorrect; I've been bitten time and again by this problem in many different contexts, enough to know that it's not obvious what the “right” solution is. -- \ “Not to perambulate the corridors in the hours of repose in the | `\ boots of ascension.” —ski hotel, Austria | _o__) | Ben Finney ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 5/08/2009 6:00 PM, Ben Finney wrote: Mark Hammond writes: As already mentioned in this thread, a capability similar to what svn or cvs offers would be sufficient. That capability presented by centralised VCSen is entirely dependent on the fact that they *are* centralised. Using a distributed VCS means the same capability doesn't apply. Why do you say that (without justification I might add ) about this issue? While a DVCS does offer unique challenges, it seems to me that doing something at commit time without requiring magic hooks be configured would go a long way to addressing the problem. The hand-waving “doing something” is exactly what needs to be solved. I think you have been mis-reading this thread. It is quite clear what 'doing something' means in this context - it means implement the human-defined rules for the line-ending policy for the repository. Magic hooks on the official repo would then be considered the final fallback defense, but should rarely be invoked. Right, so that's “capability similar to centralised VCS” out of consideration; I'm glad we agree in the end. I'm afraid you have lost me again, as clearly we don't agree on what useful things can be done at local commit time. I'm not sure what point you are trying to make That I disagree with your position. You seem to think that the problem has an obvious solution, which is not true; and that choice of a distributed VCS should be delayed until the problem is solved, which I don't agree with. Fair enough - but it seems clear to enough of us that we can make progress and meet the requirements of the people actually impacted. but I believe it *is* possible for a solution to be found here which will keep Windows users happy. I'm guessing you haven't had much practical experience with this problem, so probably don't see this is clearly as Windows users do. Your guess is incorrect; I've been bitten time and again by this problem in many different contexts, enough to know that it's not obvious what the “right” solution is. Sorry about that - but that was the only way I could explain you not seeing how such a solution can work. Cheers, Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
>> Now, the specific outcome of the process means that more work needs to >> be done. So we have a *second* PEP, and we have a lack of volunteers >> that help implementing it. The second PEP hasn't been approved yet >> (as it isn't complete, yet), so migration to hg is stalled. >> The primary volunteer (Dirkjan) has indicated that he can't help with >> that specific issue, so other volunteers need to step forward, or we >> cannot move to hg. > > I don't recall Dirkjan saying he can't help with that issue - was it a > lack of time, or a lack of understanding the problem/lack of a Windows > environment? I think he said (at some point) that he is not a Windows user, and thus can't really help. Of course, he also indicated that, as a Mercurial contributor, he is willing to help as much as he can. > The problem I see is a lack of agreement about exactly what the solution > entails. I believe there is general agreement win32text needs to be > enhanced to support versioned 'rules'. But even with that, the only > option I see is a truly cross-platform extension to implement these > rules which every Python committer, regardless of operating-system, is > expected to use - but that doesn't seem the consensus. > > As mentioned, I'm willing to lend manpower for this once there is > agreement on something workable... I think it needs to work the other way 'round. Somebody (perhaps you) needs to propose a hook and configuration settings, and propose that this hook is used on every system, and that refusal to use these hooks could lead to changes not being integratable (is that a word?). There can't be consensus to use a solution that doesn't exist. My personal favorite outcome would be this: - most files have svn's "native" eol style; they get stored in LF in the repository; the hook will convert them on Windows, and check on Unix. - some files have "windows" eol style; they get stored in CRLF. The hook will not convert, but only check. - not sure whether some files need to be declared as "unix" eol style. - some files are "binary"; they get stored as-is - the hook will do nothing. With such a setup, using the hook would be truly optional on Unix, as it only ever checks and never converts. So if you manage to mess up, and don't have the hook installed on Unix, you lose when trying to push. That will teach you to be more careful in the future, or to install the hook (which hopefully becomes built into Mercurial at some point). Whether it is actually possible to implement all that, I don't know. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
>> As already mentioned in this thread, a capability similar to what svn >> or cvs offers would be sufficient. > > That capability presented by centralised VCSen is entirely dependent on > the fact that they *are* centralised. Using a distributed VCS means the > same capability doesn't apply. Why do you say that? People have demonstrated the contrary already. >> I'm not sure what point you are trying to make > > That I disagree with your position. You seem to think that the problem > has an obvious solution, which is not true; and that choice of a > distributed VCS should be delayed until the problem is solved, which I > don't agree with. But is *has* an obvious solution. See the implementation from Dj Gilcrease, or the spec that I just posted. > Your guess is incorrect; I've been bitten time and again by this problem > in many different contexts, enough to know that it's not obvious what > the “right” solution is. The configuration options of svn have served us well enough. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Martin v. Löwis: > Is it really that you don't *understand*? It's fairly easy: there was > a PEP ... The PEP process is straightforward. However, a PEP may produce an outcome that proves after more experience to be wrong. ISTM a prerequisite to choosing a DVCS is that it should support the full range of development platforms and thus the PEP was accepted prematurely. At some point the PEP should be reexamined and, if necessary, rescinded. What I don't understand is why the plan is still to move to hg despite, after several months, there not being a known good way to include Windows eol support. Neil ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On Wed, Aug 5, 2009 at 01:43, Mark Hammond wrote: > Thanks Nick; I didn't want to be the only one saying that. There is a fine > line between asserting reasonable requirements for Windows users and being > obstructionist and unhelpful, and I'm trying to stay on the former side :) I'm not trying to be obstructionist and unhelpful (I hope that should be obvious). On the other hand, I'm working from the point of view of hg, which has two assumptions: - we're a distributed system, there's fairly little we can assume about clients - we exchange checksummed byte streams (even if we have some tools that assume those streams are code) - because of the previous point, there's one native (and therefore better, in a sense) serialization of what you consider "structured" data The first point means, for example, there will always be some clients who don't have win32text enabled, no matter what, so you can't rely on it, which is why I want to make the server hooks the primary line of defense, and view the client-side tools as helper tools (to make it easy not to trigger the server-side hooks). That doesn't mean I think Windows users are second-rate, or anything like that! > I'm not that happy with the server being the primary line of defense. Let's > say I make a branch of the hg repo, myself and a few others work on it > committing as we go, then attempt to merge back upstream. Let's say some of > the early commits on that clone introduced "bad" line endings. I'm guessing > I would be forced to make a number of whitespace-only checkins to normalize > the line-endings before it could merge - and these checkins would then be in > the history forever. Or I could attempt to recreate the clone by somehow > "replaying" the commits with line endings corrected. Either way, the > situation doesn't seem good. I don't think either is bad. In the first case, you have one or maybe two extra changesets. As we like to advocate small changesets that fix one thing, a changeset fixing up whitespace is par for the course. ;) The other solution would be to employ mq, for example, to fix up the commits, which mq excels at (although admittedly it has a learning curve). > I agree. It isn't fair to make this windows users problem. It would be > like me proposing the repo get imported with \r\n line endings, enforce that > with server side hooks, and let non-Windows users worry about the > ramifications of that - somehow I doubt that would fly - so neither should > it fly for Windows users... > > I'm more than willing to help on this; I haven't resurrected my stale patch > because I find win32text only 1/2 a solution that doesn't work in practice. > Therefore that patch is as stale for me as it is anyone. However, if a plan > is put in place which offers a full solution and the hg developers are > committed to it, I promise I'll put my hand up to help with implementation > in a fairly timely manner... Well, I'd be happy to help convince the hg crew to accept whatever we come up with, but I'm not sure I'm the best person to come up with it. It sounds like a versioned .hgeols would help a bunch of issues, but I have the feeling you know that better than me, so I'm hoping you can come up with a concrete proposal on what should change in win32text to fix all the problems you see. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
>The PEP process is straightforward. However, a PEP may produce an > outcome that proves after more experience to be wrong. ISTM a > prerequisite to choosing a DVCS is that it should support the full > range of development platforms and thus the PEP was accepted > prematurely. To be as blunt as possible: the PEP was accepted because Guido really, Really, REALLY wanted to switch to Mercurial. So you would have to convince Guido to revert his decision. You may not like the decision (I did not like using a DVCS in the first place), but following such decisions has served us well, and will serve us well this time. > At some point the PEP should be reexamined and, if > necessary, rescinded. What I don't understand is why the plan is still > to move to hg despite, after several months, there not being a known > good way to include Windows eol support. You don't understand why it takes many months? That's also easy: because there is a single volunteer, and because there is a lot of work. I think it took me a year to migrate to subversion back then, and I wouldn't be surprised if the Mercurial migration takes even longer. Or don't you understand why that single unresolved item didn't manage to revert the decision? Well, there are many unresolved items in the Mercurial conversion, some much more stressful than the eol issue (e.g. the branching discussion). None of them is unsolvable (AFAICT); you can either contribute to the solution, and sit back and wait for solutions to emerge. Then you can vote on PEP 385 up or down still. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
> - we're a distributed system, there's fairly little we can assume about > clients Not as Mercurial, no. As Python, we can certainly expect that all of our contributors have read the developer FAQ, and set up their systems accordingly. If all else fails, we can revoke commit access (or is it "push access"?) if some committer doesn't get the configuration right. We would, of course, prefer if it was very easy to get the configuration right, so that problems don't occur in the first place. > The first point means, for example, there will always be some clients > who don't have win32text enabled, no matter what, so you can't rely on > it, which is why I want to make the server hooks the primary line of > defense I think it's a terminology issue only: don't say "primary", say "last". Can we agree that the "last" line of defense will be the server hooks, and the "primary" line of defense will be the client commits? "primary" would mean that this is were most errors are detected and fixed; Mark would really object to a flow where most errors are detected only at the server. > That doesn't mean I think > Windows users are second-rate, or anything like that! If the server hooks were the primary line of defense, it would effectively make Windows users second-rate: they will have to redo all their changes over-and-over again, whereas the Unix users can push the changes without any obstacles (just because they are less likely to make mistakes). If the client machines were the primary line of defense, Windows users were treated equally: they would make as few mistakes as Unix users, because the hooks do what they want correctly. > I don't think either is bad. In the first case, you have one or maybe > two extra changesets. As we like to advocate small changesets that fix > one thing, a changeset fixing up whitespace is par for the course. ;) Whitespace-only changes hurt the "annotate" feature, so we dislike them very much in Python. > Well, I'd be happy to help convince the hg crew to accept whatever we > come up with, but I'm not sure I'm the best person to come up with it. That is all very well. See my other message (asking for volunteers) as well. If you have more work you would prefer to delegate, please let us know. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 5/08/2009 6:25 PM, Dirkjan Ochtman wrote: On Wed, Aug 5, 2009 at 01:43, Mark Hammond wrote: Thanks Nick; I didn't want to be the only one saying that. There is a fine line between asserting reasonable requirements for Windows users and being obstructionist and unhelpful, and I'm trying to stay on the former side :) I'm not trying to be obstructionist and unhelpful (I hope that should be obvious). It is, and I hope I didn't imply otherwise. On the other hand, I'm working from the point of view of hg, which has two assumptions: - we're a distributed system, there's fairly little we can assume about clients - we exchange checksummed byte streams (even if we have some tools that assume those streams are code) - because of the previous point, there's one native (and therefore better, in a sense) serialization of what you consider "structured" data The first point means, for example, there will always be some clients who don't have win32text enabled, no matter what, so you can't rely on it, which is why I want to make the server hooks the primary line of defense, and view the client-side tools as helper tools (to make it easy not to trigger the server-side hooks). That doesn't mean I think Windows users are second-rate, or anything like that! In general I agree - although I think we can enforce a "social contract" which puts requirements on people who commit to the Python repository - and therefore we can consider the server-side hooks a "secondary" defense. IOW, the system (including the social aspects of the system) are setup such that the server-side hooks are very rarely called upon. I'm not that happy with the server being the primary line of defense. Let's say I make a branch of the hg repo, myself and a few others work on it committing as we go, then attempt to merge back upstream. Let's say some of the early commits on that clone introduced "bad" line endings. I'm guessing I would be forced to make a number of whitespace-only checkins to normalize the line-endings before it could merge - and these checkins would then be in the history forever. Or I could attempt to recreate the clone by somehow "replaying" the commits with line endings corrected. Either way, the situation doesn't seem good. I don't think either is bad. With all due respect, I suspect that is because you don't expect to see the issue regularly. This proposal still leaves the problem squarely in the lap of Windows users and imposes a burden on them that would probably be considered unreasonable if the situation was reversed. I'm yet to work on a hg repository without mixed line endings. If I understand correctly, every such repository would have involved a developer checking in locally, than at some point in the future pushing these changes upstream. I really really don't want hg to tell me at this final step that I need to perform whitespace only fixes purely because I am running Windows. I understand we are discussing how win32text can offer that - but I must object to your assertion that the situation I described isn't bad when you hit it. Well, I'd be happy to help convince the hg crew to accept whatever we come up with, but I'm not sure I'm the best person to come up with it. It sounds like a versioned .hgeols would help a bunch of issues, but I have the feeling you know that better than me, so I'm hoping you can come up with a concrete proposal on what should change in win32text to fix all the problems you see. Actually, I think it is easy to make this problem much easier to understand; mandate every platform should use win32text, then start collating the issues people, including yourself, will no doubt face. I'm happy to get this ball rolling, but again, don't want this left purely in the domain of "it is a windows problem" - it isn't. Cheers, Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On Wed, Aug 5, 2009 at 10:51, "Martin v. Löwis" wrote: > Not as Mercurial, no. As Python, we can certainly expect that all of our > contributors have read the developer FAQ, and set up their systems > accordingly. If all else fails, we can revoke commit access (or is > it "push access"?) if some committer doesn't get the configuration > right. We would, of course, prefer if it was very easy to get the > configuration right, so that problems don't occur in the first place. There will also be non-committers who forge changesets that you want to be able to push directly to the Python repositories. > If the client machines were the primary line of defense, Windows users > were treated equally: they would make as few mistakes as Unix users, > because the hooks do what they want correctly. Similarly, if Python kept its .py files in \r\n line endings by default instead of \n endings, Unix-like users would be more prone to mistake, so by keeping the .py files in \n-format, so Python is making Windows users second-rate by keeping the line endings in \n format. To cope with that, hg needs to do extra work on the client side. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Martin v. Löwis: > Or don't you understand why that single unresolved item didn't manage > to revert the decision? Well, there are many unresolved items in > the Mercurial conversion, some much more stressful than the eol issue > (e.g. the branching discussion). Then these issues should have been included in the initial PEP for choosing a DVCS since the issues could have driven the choice. PEP 374 implies that win32text effectively solves the Windows eol issue which no longer appears to be correct. Neil ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On Wed, Aug 5, 2009 at 11:02, Mark Hammond wrote: > In general I agree - although I think we can enforce a "social contract" > which puts requirements on people who commit to the Python repository - and > therefore we can consider the server-side hooks a "secondary" defense. IOW, > the system (including the social aspects of the system) are setup such that > the server-side hooks are very rarely called upon. Agreed. > With all due respect, I suspect that is because you don't expect to see the > issue regularly. I suspect so, too! > I'm yet to work on a hg repository without mixed line endings. If I > understand correctly, every such repository would have involved a developer > checking in locally, than at some point in the future pushing these changes > upstream. I really really don't want hg to tell me at this final step that > I need to perform whitespace only fixes purely because I am running Windows. > > I understand we are discussing how win32text can offer that - but I must > object to your assertion that the situation I described isn't bad when you > hit it. I agree it is to be avoided, I'm just saying that I think it will be exceptional and therefore not a large burden, given other kinds of defenses we can put in place. > Actually, I think it is easy to make this problem much easier to understand; > mandate every platform should use win32text, then start collating the issues > people, including yourself, will no doubt face. I'm happy to get this ball > rolling, but again, don't want this left purely in the domain of "it is a > windows problem" - it isn't. I'm not sure how win32text will provide anything other than performance degradation for non-Windows developers, but if there's functionality to be had, I'm happy to mandate its use on every platform. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
>> Not as Mercurial, no. As Python, we can certainly expect that all of our >> contributors have read the developer FAQ, and set up their systems >> accordingly. If all else fails, we can revoke commit access (or is >> it "push access"?) if some committer doesn't get the configuration >> right. We would, of course, prefer if it was very easy to get the >> configuration right, so that problems don't occur in the first place. > > There will also be non-committers who forge changesets that you want > to be able to push directly to the Python repositories. They will also have to follow the policies we set up. If they refuse to do that, we refuse to accept their changes. It's very simple, and contributors have learned very quickly what the policies were (after they were explained to them). Whether that means that they have to fix their changesets, or that they have to redo them, practice will show. >> If the client machines were the primary line of defense, Windows users >> were treated equally: they would make as few mistakes as Unix users, >> because the hooks do what they want correctly. > > Similarly, if Python kept its .py files in \r\n line endings by > default instead of \n endings, Unix-like users would be more prone to > mistake, so by keeping the .py files in \n-format, so Python is making > Windows users second-rate by keeping the line endings in \n format. To > cope with that, hg needs to do extra work on the client side. I think you still miss the point. *If* hg does the extra work, *then* Windows users are *not* second-class citizens anymore. They *only* consider themselves second-class if they have to do additional *manual* work (*). Regards, Martin (*) They may also consider themselves second-class if they have to install additional software, so hopefully, the necessary extra code for hg will become part of the regular Mercurial distribution at some point. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
> I'm not sure how win32text will provide anything other than > performance degradation for non-Windows developers, but if there's > functionality to be had, I'm happy to mandate its use on every > platform. This is all fairly hypothetical - if hg grew a .hgeols file, it would be good if it supported that cross-platform. It then may make win32text obsolete (in particular if it provided some useful defaults). On Unix, the functionality might be as simple as checking conformance with the eol-style at pre-commit time. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 5/08/2009 7:09 PM, Dirkjan Ochtman wrote: I'm not sure how win32text will provide anything other than performance degradation for non-Windows developers, but if there's functionality to be had, I'm happy to mandate its use on every platform. I see two practical outcomes of such a mandate: * line-ending rules are enforced for local checkins, even for linux users, even though such 'accidental' inappropriate line-ending checkins should be much rarer than for windows. * practical problems faced by Windows users, including any performance considerations, are shared by the community and therefore addressed as a community, thereby ensuring all platforms are considered as important as any other. Cheers, Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
"Martin v. Löwis" writes: > > You seem to think that the problem has an obvious solution, which is > > not true; > But is *has* an obvious solution. See the implementation from Dj > Gilcrease, or the spec that I just posted. Two different solutions are both obvious? There are other solutions proposed elsewhere too; are they also obvious? Mark Hammond writes: > I think you have been mis-reading this thread. Quite possibly; I'm not intending to impose my position on anyone. I'll go back to lurking on the thread for a while and see if it becomes any clearer. -- \ “First things first, but not necessarily in that order.” —The | `\ Doctor, _Doctor Who_ | _o__) | Ben Finney ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
2009/8/5 "Martin v. Löwis" : > My personal favorite outcome would be this: > - most files have svn's "native" eol style; they get stored in LF > in the repository; the hook will convert them on Windows, and check > on Unix. > - some files have "windows" eol style; they get stored in CRLF. > The hook will not convert, but only check. > - not sure whether some files need to be declared as "unix" eol style. > - some files are "binary"; they get stored as-is - the hook will > do nothing. > > With such a setup, using the hook would be truly optional on Unix, > as it only ever checks and never converts. So if you manage to mess > up, and don't have the hook installed on Unix, you lose when trying > to push. That will teach you to be more careful in the future, or > to install the hook (which hopefully becomes built into Mercurial at > some point). Given that my preference is to use Unix-style EOL for "text" files on Windows, as every text editor I use (barring notepad!) understands LF format, it seems to me that this proposal also means that the hook would be optional for me. That suits me fine - I'd prefer to avoid having hooks that are required for Python checkouts, as that means I have to remember to configure them on each clone (IIUC). Of course, this implies that your proposal only requires any action by the user in the case of Windows users whose text editing tools insist on CRLF format text files (sources, etc). Is that really a large group of developers? (I honestly don't know). I suspect that there is something missing from your proposal, as if this were the case, then the problem appears to be limited to a very small group of developers. Maybe it's Visual Studio that insists on CRLF for source files? (I don't know, as I don't use the VS editor). If that's the case, then maybe a VS hook would be an alternative approach? (I can't imagine such a hook would be an *easier* approach, I only mention it because it makes it clearer where the issue lies). Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On Wed, Aug 5, 2009 at 12:04, Paul Moore wrote: > Given that my preference is to use Unix-style EOL for "text" files on > Windows, as every text editor I use (barring notepad!) understands LF > format, it seems to me that this proposal also means that the hook > would be optional for me. That suits me fine - I'd prefer to avoid > having hooks that are required for Python checkouts, as that means I > have to remember to configure them on each clone (IIUC). Yeah, this may also be what's making it harder for me to understand the issues. I am actually a Windows user, although I do most of my development on Linux servers through PuTTY. I just always make sure I use editors that respect the file's line endings, and so for those things where I've used hg to version code on Windows (for example, when testing a Firefox extension) and when my colleague who does edit his code inside Windows, I've just used editors that deal with line endings. Typically, in my case, that was either Notepad2 (an awesomely light-weight Notepad replacement) or Komodo (Edit). That solved all of my issues, so I haven't had a need for win32text so far. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Functionality in subprocess.Popen.terminate()
Janzert wrote: > Eric Pruitt wrote: >> Sounds good enough to me but I was wondering if it might be a good >> idea to add a function like "pidinuse" to subprocess as a whole that >> would determine if a process ID was being used and return a simple >> boolean value. I came across a number of people searching for a way to >> determine if a PID was running (Google "python check if pid exists") >> so it seems like the implemented functionality would be of use to the >> community as a whole, not just my wrapper class. >> >> Eric >> > > I'm not sure of the actual details but it seems from your description > that even if you check first a race condition will still exist. > Specifically the subprocess could terminate after the check and before > the TerminateProcess call. So it seems better just to call > TerminateProcess and then correctly handle any possible error. Janzert is correct here - this is a case where ruling out the error completely is impossible, so you're going to have to handle it regardless. A cross platform way of checking if a particular subprocess is still running might be an interesting feature in its own right, but I don't think it will prevent this exception. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 5/08/2009 8:04 PM, Paul Moore wrote: 2009/8/5 "Martin v. Löwis": With such a setup, using the hook would be truly optional on Unix, as it only ever checks and never converts. So if you manage to mess up, and don't have the hook installed on Unix, you lose when trying to push. That will teach you to be more careful in the future, or to install the hook (which hopefully becomes built into Mercurial at some point). Given that my preference is to use Unix-style EOL for "text" files on Windows, as every text editor I use (barring notepad!) understands LF format, Most tools that I use will tend to not mix EOL styles in a single file, but will tend to create \r\n line endings for new files I create. Most hg repos I come across don't have mixed line endings within individual files, so I can only guess these files were accidentally introduced in the same way (and indeed I have personally done this.) I'm hoping to be part of the solution instead of part of the problem :) > it seems to me that this proposal also means that the hook would be optional for me. Technically it would be optional for everyone, of course. However, the solution should be such that everyone, regardless of personal preference, is willing to take the hit. For example, if the repo is converted using \r\n line endings natively, then Windows users would need to take no action either and puts the onus back on you (given your stated preferences) to configure the tool appropriately. I assume you would have no objection to that and would be happy to make that tool optional for me? That suits me fine - I'd prefer to avoid having hooks that are required for Python checkouts, as that means I have to remember to configure them on each clone (IIUC). Configuring on each clone would certainly be sub-optimal, so the proposal is this configuration be stored in a versioned file in the repo. Of course, this implies that your proposal only requires any action by the user in the case of Windows users whose text editing tools insist on CRLF format text files (sources, etc). Is that really a large group of developers? (I honestly don't know). It applies to all files that aren't "native" EOL style - there are just less of them regularly modified than those that are so marked. I suspect that there is something missing from your proposal, as if this were the case, then the problem appears to be limited to a very small group of developers. Maybe it's Visual Studio that insists on CRLF for source files? (I don't know, as I don't use the VS editor). If that's the case, then maybe a VS hook would be an alternative approach? (I can't imagine such a hook would be an *easier* approach, I only mention it because it makes it clearer where the issue lies). I must concede that Windows developers are the minority here - but assuming we want a level playing field, I don't see how that changes the underlying issue... Cheers, Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 5/08/2009 8:14 PM, Dirkjan Ochtman wrote: endings. Typically, in my case, that was either Notepad2 (an awesomely light-weight Notepad replacement) or Komodo (Edit). That solved all of my issues, so I haven't had a need for win32text so far. FWIW, I use komodo and scite as my primary editors, and as mentioned, am personally responsible for accidentally checking in \r\n files into what should be a \n repo. I am slowly and painfully learning to be more careful - IMO, I shouldn't need to... Cheers, Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On Wed, Aug 5, 2009 at 13:19, Mark Hammond wrote: > Configuring on each clone would certainly be sub-optimal, so the proposal is > this configuration be stored in a versioned file in the repo. Even if we do that, enabling hg extensions will still need to be done locally -- although it can be done per-user/box instead of per-clone. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 5/08/2009 9:28 PM, Dirkjan Ochtman wrote: On Wed, Aug 5, 2009 at 13:19, Mark Hammond wrote: Configuring on each clone would certainly be sub-optimal, so the proposal is this configuration be stored in a versioned file in the repo. Even if we do that, enabling hg extensions will still need to be done locally -- although it can be done per-user/box instead of per-clone. That is completely fine, and not unlike SVN where a per-user/box setting generally needs to be set once - but after that everything "just works". Windows developers don't mind taking a hit once ;) The dev guide can make it clear what the expectations are... Cheers, Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Mark Hammond wrote: > On 5/08/2009 7:09 PM, Dirkjan Ochtman wrote: >> I'm not sure how win32text will provide anything other than >> performance degradation for non-Windows developers, but if there's >> functionality to be had, I'm happy to mandate its use on every >> platform. > > I see two practical outcomes of such a mandate: > > * line-ending rules are enforced for local checkins, even for linux > users, even though such 'accidental' inappropriate line-ending checkins > should be much rarer than for windows. > > * practical problems faced by Windows users, including any performance > considerations, are shared by the community and therefore addressed as a > community, thereby ensuring all platforms are considered as important as > any other. The main error that enabling win32text everywhere can catch is the use of a *nix client to accidentally corrupt one of the files that is supposed to have \r\n line endings. It also simplifies the configuration rules in the Python hg FAQ - we would be able to just tell all developers wanting to contribute patches to Python to enable the win32text extension when working with the Python repositories (or clones thereof) without having to worry about what platform they were on. So it seems to me that the main client-side feature we want is a versioned .hgeols file in the repository that allows files to be explicitly nominated as one of: - eol=CRLF (i.e. have \r\n line endings in the repository and should be left that way on the local disk as well - equivalent to SVN eol-style:CRLF) - eol=LF (i.e. have \n line endings in the repository and should be left that way on the local disk as well - equivalent to SVN eol-style:LF) - eol=CR (i.e. have \n line endings in the repository and should be left that way on the local disk as well - equivalent to SVN eol-style:CR) - native text (i.e. always stored in the repository with \n line endings, but uses native line endings on the local disk - equivalent to SVN eol-style:native) - binary (i.e. always reproduced on disk exactly as they are in the repository - equivalent to SVN files without eol-style set at all) The .hgeols file should also allow the repository to define which of the above should be used as the default handling mechanism for text files that are not named in the file (native text, in the specific case of the Python repositories). Files which look like binary files (according to the existing win32text heuristics) would be left alone regardless of what the default handling was set to in .hgeols. win32text would then be enhanced to check for a .hgeols file before falling back to its existing configuration mechanisms. The above basically provides the SVN eol-style feature in a more hg-friendly way. Allowing wildcards in the .hgeols files might be nice, but I don't think it is actually required. We really don't have that many files that are affected by this problem (it's just the fact that it is a number greater than zero that is causing the problem). The server side pre-push hooks for the main Python repositories would be set to reject change sets which didn't meet the above rules. If a patch fails those checks, either the committer can fix it themselves and resubmit, or else send it back to the originator along with a pointer to the section in the dev FAQ that describes the expected client-side configuration. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Nick Coghlan wrote: Mark Hammond wrote: On 5/08/2009 7:09 PM, Dirkjan Ochtman wrote: I'm not sure how win32text will provide anything other than performance degradation for non-Windows developers, but if there's functionality to be had, I'm happy to mandate its use on every platform. I see two practical outcomes of such a mandate: * line-ending rules are enforced for local checkins, even for linux users, even though such 'accidental' inappropriate line-ending checkins should be much rarer than for windows. * practical problems faced by Windows users, including any performance considerations, are shared by the community and therefore addressed as a community, thereby ensuring all platforms are considered as important as any other. The main error that enabling win32text everywhere can catch is the use of a *nix client to accidentally corrupt one of the files that is supposed to have \r\n line endings. It also simplifies the configuration rules in the Python hg FAQ - we would be able to just tell all developers wanting to contribute patches to Python to enable the win32text extension when working with the Python repositories (or clones thereof) without having to worry about what platform they were on. So it seems to me that the main client-side feature we want is a versioned .hgeols file in the repository that allows files to be explicitly nominated as one of: - eol=CRLF (i.e. have \r\n line endings in the repository and should be left that way on the local disk as well - equivalent to SVN eol-style:CRLF) - eol=LF (i.e. have \n line endings in the repository and should be left that way on the local disk as well - equivalent to SVN eol-style:LF) - eol=CR (i.e. have \n line endings in the repository and should be left that way on the local disk as well - equivalent to SVN eol-style:CR) - native text (i.e. always stored in the repository with \n line endings, but uses native line endings on the local disk - equivalent to SVN eol-style:native) - binary (i.e. always reproduced on disk exactly as they are in the repository - equivalent to SVN files without eol-style set at all) The .hgeols file should also allow the repository to define which of the above should be used as the default handling mechanism for text files that are not named in the file (native text, in the specific case of the Python repositories). Files which look like binary files (according to the existing win32text heuristics) would be left alone regardless of what the default handling was set to in .hgeols. win32text would then be enhanced to check for a .hgeols file before falling back to its existing configuration mechanisms. The above basically provides the SVN eol-style feature in a more hg-friendly way. Allowing wildcards in the .hgeols files might be nice, but I don't think it is actually required. We really don't have that many files that are affected by this problem (it's just the fact that it is a number greater than zero that is causing the problem). The server side pre-push hooks for the main Python repositories would be set to reject change sets which didn't meet the above rules. If a patch fails those checks, either the committer can fix it themselves and resubmit, or else send it back to the originator along with a pointer to the section in the dev FAQ that describes the expected client-side configuration. Instead of just talking about line endings, could each file have a specific 'filetype'? This would define what kind of data it contains, how it's stored in the repository, and what actions to perform for fetching and committing, including any checks: c_header: C header file; LF in repository; native outside c_source: C source file; LF in repository; native outside text: plain text; LF in repository; native outside crlf_text: plain text; CRLF in repository; CRLF outside cr_text: plain text; CR in repository; CR outside lf_text: plain text; LF in repository; LF outside binary: arbitrary binary data; as-is in repository This could be expanded in the future to include filetypes for JPEG, etc. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On Wed, Aug 5, 2009 at 15:35, MRAB wrote: > Instead of just talking about line endings, could each file have a > specific 'filetype'? This would define what kind of data it contains, > how it's stored in the repository, and what actions to perform for > fetching and committing, including any checks: Sounds like YAGNI to me. The outline Nick provided seems to me to be quite close to the current win32text settings in syntax and purpose and staying close to that would help making adoption easier. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On Wed, Aug 05, 2009 at 02:35:02PM +0100, MRAB wrote: > Instead of just talking about line endings, could each file have a > specific 'filetype'? EOL-conversion, MIME type and encoding (charset) are three different concepts. Yes, all of them must be supported, but not necessary in one configuration mechanism. Subversion handles these issues by providing svn:eol-style and svn:mime-type (handles both MIME type and charset) properties on a file-by-file basis. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
On Wed, Aug 05, 2009 at 05:50:03PM +0400, Oleg Broytmann wrote: >Subversion handles these issues by providing ... > svn:mime-type (handles both MIME type and charset) > file-by-file basis. Dirkjan, how does Mercurial handles charsets? If I have three files in my repository - one in utf-8, another in koi8-r, and the third in cp1251 encoding - I certainly don't want to convert them back and force, but I want hg web interface to provide charset in the Content-Type header. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
On Wed, Aug 5, 2009 at 15:57, Oleg Broytmann wrote: > Dirkjan, how does Mercurial handles charsets? If I have three files in > my repository - one in utf-8, another in koi8-r, and the third in cp1251 > encoding - I certainly don't want to convert them back and force, but I > want hg web interface to provide charset in the Content-Type header. It doesn't currently have any way to provide out-of-band charset info. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Dirkjan Ochtman wrote: > On Wed, Aug 5, 2009 at 15:35, MRAB wrote: >> Instead of just talking about line endings, could each file have a >> specific 'filetype'? This would define what kind of data it contains, >> how it's stored in the repository, and what actions to perform for >> fetching and committing, including any checks: > > Sounds like YAGNI to me. Yep - while SVN does support full mime_type specification for files, I don't think we have ever used it. The SVN eol-style property is all we're trying to replicate, since that has served us well in the few cases where it has mattered. > The outline Nick provided seems to me to be > quite close to the current win32text settings in syntax and purpose > and staying close to that would help making adoption easier. Yeah, win32text is already tantalising close to what we would like so I deliberately tried to stay close to its existing approach. We're just being a bit fussier than most about the repository being able to tell the clients which files should be given special treatment. That way individual users can just set it up once on their development machine and then no longer have to worry about it (if more files that need special treatment are added to the repository, then the same checkin that adds them should also update .hgeols). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Mark Hammond writes: > I'm not sure what point you are trying to make, but I believe it *is* > possible for a solution to be found here which will keep Windows users > happy. I'm guessing you haven't had much practical experience with this > problem, so probably don't see this is clearly as Windows users do. Mercurial is not only open source, it's written in Python. The problem is known to be hard in a practical sense, the existing solutions (written by non-Windows developers, of course) are judged to be insufficient by Windows users, and the non-Windows developers "probably don't see this is clearly as Windows users do." I think the implication is obvious. There will be no good solution until Windows users develop it. I don't see a good reason to wait for that. I do see good reason for non-Windows users to put up with some inconvenience during the "beta" phase of implementing that solution; it's important enough to be fast-tracked, and doesn't need to be perfect for everybody to be tried (though it should not be allowed to endanger repo content, which seems unlikely but needs care since it's a potential disaster). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
On Wed, Aug 05, 2009 at 04:04:24PM +0200, Dirkjan Ochtman wrote: > On Wed, Aug 5, 2009 at 15:57, Oleg Broytmann wrote: > > Dirkjan, how does Mercurial handles charsets? If I have three files in > > my repository - one in utf-8, another in koi8-r, and the third in cp1251 > > encoding - I certainly don't want to convert them back and force, but I > > want hg web interface to provide charset in the Content-Type header. > > It doesn't currently have any way to provide out-of-band charset info. Perhaps that's not a big issue for Python, but it's certainly a big issue for me. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
On Wed, Aug 5, 2009 at 16:35, Oleg Broytmann wrote: > Perhaps that's not a big issue for Python, but it's certainly a big > issue for me. I think there are extensions that try to deal with it. Have a look: http://mercurial.selenic.com/wiki/UsingExtensions If not, it should be easy to come up with something and write an extension for it. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the charset issue
On Thu, Aug 06, 2009 at 12:12:08AM +1000, Nick Coghlan wrote: > Yep - while SVN does support full mime_type specification for files, I > don't think we have ever used it. These files are in 8859-1 encoding (names in comments, at least): http://svn.python.org/view/python/trunk/Lib/encodings/punycode.py http://svn.python.org/view/python/trunk/Lib/test/test_csv.py http://svn.python.org/view/python/trunk/Tools/i18n/msgfmt.py http://svn.python.org/view/python/trunk/Tools/i18n/pygettext.py If they are not marked as "text/plain; charset=iso-8859-1" I think it's a bug. Either they should be marked, or converted to ascii or utf-8; the coding pseudocomment (directive) should be changed accordingly. Probably there are other files. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Mark Hammond wrote: > On 5/08/2009 8:14 PM, Dirkjan Ochtman wrote: >> endings. Typically, in my case, that was either Notepad2 (an awesomely >> light-weight Notepad replacement) or Komodo (Edit). That solved all of >> my issues, so I haven't had a need for win32text so far. > > FWIW, I use komodo and scite as my primary editors, and as mentioned, am > personally responsible for accidentally checking in \r\n files into what > should be a \n repo. I am slowly and painfully learning to be more > careful - IMO, I shouldn't need to... > > Cheers, > > Mark IIRC one of the main problems in Copy & Paste. I believe both Scite and Visual Studio have had issues where they "preserve" the line endings of files, but if you paste from another source, it will continue to "preserve" the line endings of the pasted content. That said, you also have the "create a new file defaults to CRLF" that has similar problems. John =:-> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the charset issue
Oleg Broytmann phd.pp.ru> writes: > >These files are in 8859-1 encoding (names in comments, at least): > http://svn.python.org/view/python/trunk/Lib/encodings/punycode.py > http://svn.python.org/view/python/trunk/Lib/test/test_csv.py > http://svn.python.org/view/python/trunk/Tools/i18n/msgfmt.py > http://svn.python.org/view/python/trunk/Tools/i18n/pygettext.py >If they are not marked as "text/plain; charset=iso-8859-1" I think it's > a bug. Either they should be marked, or converted to ascii or utf-8; the > coding pseudocomment (directive) should be changed accordingly. It's certainly ok to convert them to utf-8 (and add the marker anyway). There's no point in having different charsets used throughout the code base, except for testing purposes (just as there's no point in having different indentation rules used for the same file type throughout the code base ;-)). Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
Oleg Broytmann writes: >Dirkjan, how does Mercurial handles charsets? If I have three files in > my repository - one in utf-8, another in koi8-r, and the third in cp1251 > encoding - I certainly don't want to convert them back and force, but I > want hg web interface to provide charset in the Content-Type header. How is this relevant to PEP 385? I hope the answer is "not at all". I've been there, done that, and my answer is "never again". (I'm not telling you what to do with *your* repository, just that I don't see any good reason for having any encodings but UTF-8 in Python's.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
On Thu, Aug 06, 2009 at 12:34:39AM +0900, Stephen J. Turnbull wrote: > Oleg Broytmann writes: > >Dirkjan, how does Mercurial handles charsets? If I have three files in > > my repository - one in utf-8, another in koi8-r, and the third in cp1251 > > encoding - I certainly don't want to convert them back and force, but I > > want hg web interface to provide charset in the Content-Type header. > > How is this relevant to PEP 385? I hope the answer is "not at all". There are non-utf8 non-ascii files in the Python source tree. Either there should be a way to handle them in Mercurial or they have to be converted to UTF-8 in a proper way (i.e., don't forget to rewrite charset directives). Other tan that - I am pondering a switch from SVN to hg in other projects using Python process as an example and asking questions that are slightly off-topic (but only slightly). > I've been there, done that, and my answer is "never again". (I'm not > telling you what to do with *your* repository, just that I don't see > any good reason for having any encodings but UTF-8 in Python's.) We have files in at least two different encodings - utf-8 and cp1251 for user-visible text-files on w32. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Microsoft MSDN
I sent fourteen requests for licenses in to Microsoft. I've asked them to let me know which they grant (since they may choose to limit the number) and will inform you all personally when I hear their decision. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Watch PyCon on video now! http://pycon.blip.tv/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
2009/8/5 Mark Hammond : > Most tools that I use will tend to not mix EOL styles in a single file, but > will tend to create \r\n line endings for new files I create. Most hg repos > I come across don't have mixed line endings within individual files, so I > can only guess these files were accidentally introduced in the same way (and > indeed I have personally done this.) I'm hoping to be part of the solution > instead of part of the problem :) Interesting. I don't recall *ever* having generated CRLF line endings in a LF-delimited file (I use Vim) although I may have created CRLF in new files (and then not noticed, as Vim handles it transparently enough that I missed it). There are no significant projects where I'm a committer, though, so I interact via patches, which means I don't get the opportunity to break the repository :-) > Technically it would be optional for everyone, of course. However, the > solution should be such that everyone, regardless of personal preference, is > willing to take the hit. > > For example, if the repo is converted using \r\n line endings natively, then > Windows users would need to take no action either and puts the onus back on > you (given your stated preferences) to configure the tool appropriately. I > assume you would have no objection to that and would be happy to make that > tool optional for me? Absolutely. My issue is with 2 points: 1) I'm an infrequent contributor, so I don't keep a checkout around. I make a new clone "on demand", so I would be likely to forget to enable the hook on at least a proportion of my clones. The versioned .hgeols proposal seems to cover this. 2) This behaviour is something needed for Python only. I've no issue with enabling win32text globally, but I'd want to be clear that it is a no-op unless specifically requested (ie, something like **=cleverencode is *not* used in the absence of an explicit set of rules). That may well be the case, but I had the impression that win32text tried to be "automatic", so I'd like to verify it. > I must concede that Windows developers are the minority here - but assuming > we want a level playing field, I don't see how that changes the underlying > issue... Again, agreed entirely. As a Windows developer who doesn't (knowingly) encounter the issue, I'm not in a good position to help, but I'm happy to contribute comments and test things. I'll be offline for a couple of weeks, though, so you may well have solved it before I can do anything :-) Paul ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On approximately 8/5/2009 4:28 AM, came the following characters from the keyboard of Dirkjan Ochtman: On Wed, Aug 5, 2009 at 13:19, Mark Hammond wrote: Configuring on each clone would certainly be sub-optimal, so the proposal is this configuration be stored in a versioned file in the repo. Even if we do that, enabling hg extensions will still need to be done locally -- although it can be done per-user/box instead of per-clone. On approximately 8/5/2009 9:24 AM, came the following characters from the keyboard of Paul Moore: > 2) This behaviour is something needed for Python only. I've no issue > with enabling win32text globally, but I'd want to be clear that it is > a no-op unless specifically requested (ie, something like > **=cleverencode is *not* used in the absence of an explicit set of > rules). That may well be the case, but I had the impression that > win32text tried to be "automatic", so I'd like to verify it. Depending on [Windows] users to configure their installation of Mercurial to work with the Python repository is lame; it will lead to new Windows contributors getting beat-up at check-in time, and make them less likely to want to contribute even the work they have already done (with wrong EOL), and much less to want to start future contributions, because some Unix Python hacker will be nasty about "Didn't you RTFM?" (Maybe not at first, but eventually). If the configuration settings have to be different per project for Windows developers using Mercurial for multiple projects, then that is also lame... Windows developers would have to keep changing their configurations, or (implied in above discussion) remember to recreate settings for each new clone or branch or whatever of the Python project. This is also error-prone, and leads to the above problem a different way. I have read this whole discussion, but want to step back and look at it from a theoretical viewpoint. A good solution would have the following characteristics: INSTALLATION) The developer should install the [D]VCS (for this discussion, Mercurial, present or future version), and attempt to access a repository (for this discussion, the Python repository, converted and configured for the chosen [D]VCS). The resultant environment should automatically be configured to work properly. If any [D]VCS extensions are required for the project, they should be automatically installed and configured, or the user given explicit instructions on how to do so, as a one-time installation step, that adversely affects no other projects for which the [D]VCS is used by that or other users of the present installation.. See below for what properly means. EOL CONFIGURATION) Each file, when added to the repository, should have a repository setting that indicates what the appropriate EOL type is for that file. The values I have heard are \n only, \r\n, platform-native, and binary. I haven't heard \r only in this discussion, but have heard it in other similar discussions, and it may be a useful setting for Mercurial to have, if the feature must be newly implemented there. I believe there are also systems that use RS to separate lines, and perhaps other things (and are there new Unicode control characters that could be used for line endings?), so it might be good to leave a few unassigned values in such a setting. I don't think any setting should be created to allow mixed line ending usage within a file, except binary. Per repository default for this setting should be available to avoid burdening the user when creating the typical type of file. ENCODING CONFIGURATION) Each file, when created, should have a repository settings that declares its character repertoire and encoding, and if it is a Unicode UTF encoding, whether or not it should have a leading BOM. In my opinion, all source code files should use a Unicode encoding, the exception being for test files that help test encoding support in internationalized environments. But the feature supports other people's opinions too. Per repository default for this setting should be available to avoid burdening the user when creating the typical type of file. CHECKOUT) Check-outs should be sensitive to the user's local environment (platform and locale settings), and non-binary files should be converted from the repository format to the local encoding and platform-specific line endings. Settings to override the line endings should be optionally available for users whose tools understand other line endings, and prefer them over the native line endings. If the characters used within a file cannot be converted losslessly to the encoding specified by the locale settings, then it should not be able to be checked out. A special override might be useful for using a lossy transformation for a read-only view of the file, at user request. CHECKIN) Check-ins, even local check-ins to local clones or branches, should automatically convert
[Python-Dev] Reasons for using expy
Hi, The expy project provides an express way to extend Python. After some careful considerations, I came up with some reasons for expy (this is not an exhaustive list): (I). WYSIWYG. The expy project enables you to write your module in Python the way your extension would be (WYSIWYG), and meanwhile write your implementation in pure C. You specify your modules, functions, methods, classes, and even their documentations the usual way of writing your Python correspondences. Then your provide your implementation to the functions/methods by returning a multi-line string. By such an arrangement, everything falls in its right place, and your extension code becomes easy to read and maintain. Also, the generated code is very human-friendly. (II). You only provide minimal information to indicate your intension of how your module/class would function in Python. So your extension is largely independent from the Python extension API. As your interaction with the Python extension API is reduced to minimal (you only care about the functionality and logic), it is then possible that your module written in expy can be independent of changes in the extension API. (III). The building and setup of your project can be automatically done with the distutil tool. In the tutorial, there are ample examples on how easily this is achieved. (IV). Very light weight. The expy tool is surprisingly light weight dispite of its powerful ability, as it is written in pure Python. There is no parser or compiler for code generation, but rather the powerful reflexion mechanism of Python is exploited in a clever way to generate human-friendly codes. Currently, generating code in C is supported, however, the implementation is well modularized and code generation in other languages such as Java and C++ should be easy. While there are already a couple of other projects trying to simply this task with different strategies, such as Cython, Pyrex and modulator, this project is unique and charming in its own way. All you need is the WYSIWYG Python file for your module extension, then expy takes care of everything else. What follows in this documentation is on how to extend Python in C using expy-cxpy: the module expy helps define your module, while module cxpy helps generate C codes for your defined module. For more information about expy, please visit its homepage at: http://expy.sf.net/ Cheers, Yingjie ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
>> Given that my preference is to use Unix-style EOL for "text" files on >> Windows, as every text editor I use (barring notepad!) understands LF >> format, it seems to me that this proposal also means that the hook >> would be optional for me. That suits me fine - I'd prefer to avoid >> having hooks that are required for Python checkouts, as that means I >> have to remember to configure them on each clone (IIUC). > > Yeah, this may also be what's making it harder for me to understand > the issues. Please trust that there are plenty of editors that get the line ending implementation wrong. I'm fairly certain that some Visual Studio versions are among them. They will recognize LF as a line ending, but add CRLF line breaks when the user presses enter. In addition, some editors (in particular notepad) choke when confronted with LF-only files. It is very annoying if you have to look at source code at somebody else's machine which doesn't have any programmer editor installed (except for Visual Studio). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the charset issue
>These files are in 8859-1 encoding (names in comments, at least): > http://svn.python.org/view/python/trunk/Lib/encodings/punycode.py > http://svn.python.org/view/python/trunk/Lib/test/test_csv.py > http://svn.python.org/view/python/trunk/Tools/i18n/msgfmt.py > http://svn.python.org/view/python/trunk/Tools/i18n/pygettext.py >If they are not marked as "text/plain; charset=iso-8859-1" I think it's > a bug. Either they should be marked, or converted to ascii or utf-8; the > coding pseudocomment (directive) should be changed accordingly. It's certainly a bug of the web page. I'm not so sure it's a bug in the files: I would claim that it's a bug in ViewCVS. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the charset issue
>>These files are in 8859-1 encoding (names in comments, at least): >> http://svn.python.org/view/python/trunk/Lib/encodings/punycode.py >> http://svn.python.org/view/python/trunk/Lib/test/test_csv.py >> http://svn.python.org/view/python/trunk/Tools/i18n/msgfmt.py >> http://svn.python.org/view/python/trunk/Tools/i18n/pygettext.py >>If they are not marked as "text/plain; charset=iso-8859-1" I think it's >> a bug. Either they should be marked, or converted to ascii or utf-8; the >> coding pseudocomment (directive) should be changed accordingly. > > It's certainly ok to convert them to utf-8 (and add the marker anyway). No, it's not. PEP 8 mandates that non-ASCII code in the Python source code is in Latin-1. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
> >Dirkjan, how does Mercurial handles charsets? If I have three files in > > my repository - one in utf-8, another in koi8-r, and the third in cp1251 > > encoding - I certainly don't want to convert them back and force, but I > > want hg web interface to provide charset in the Content-Type header. > > How is this relevant to PEP 385? I hope the answer is "not at all". > I've been there, done that, and my answer is "never again". (I'm not > telling you what to do with *your* repository, just that I don't see > any good reason for having any encodings but UTF-8 in Python's.) Just in case my previous message gets overlooked: PEP 8 mandates Latin-1 for Python 2.x source code (except for files that test PEP 263). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the charset issue
Martin v. Löwis v.loewis.de> writes: > > No, it's not. PEP 8 mandates that non-ASCII code in the Python source > code is in Latin-1. Ok, point taken. Having several encodings (and several indentation rules) certainly makes things more annoying for contributors than they should, however. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Neil Hodgson schrieb: > Martin v. Löwis: > >> Or don't you understand why that single unresolved item didn't manage >> to revert the decision? Well, there are many unresolved items in >> the Mercurial conversion, some much more stressful than the eol issue >> (e.g. the branching discussion). > >Then these issues should have been included in the initial PEP for > choosing a DVCS since the issues could have driven the choice. PEP 374 > implies that win32text effectively solves the Windows eol issue which > no longer appears to be correct. Apparently, it was the author's understanding at that time that win32text would be sufficient. Also, PEP 374 has not been written in isolation; at any time during the process people could have notified Dirkjan that this is not the case. The branching issue *has* been included in PEP 374; it is not a blocker for migration, but rather a decision has to be made between two similar, but in other ways quite different styles for converting SVN branches. I'm not aware of any other unresolved items; they may exist, but the fact that they're not discussed on this list in detail means that they are largely unimportant. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Stephen J. Turnbull schrieb: > Mark Hammond writes: > > > I'm not sure what point you are trying to make, but I believe it *is* > > possible for a solution to be found here which will keep Windows users > > happy. I'm guessing you haven't had much practical experience with this > > problem, so probably don't see this is clearly as Windows users do. > > Mercurial is not only open source, it's written in Python. The > problem is known to be hard in a practical sense, the existing > solutions (written by non-Windows developers, of course) are judged to > be insufficient by Windows users, and the non-Windows developers > "probably don't see this is clearly as Windows users do." > > I think the implication is obvious. There will be no good solution > until Windows users develop it. I don't see a good reason to wait for > that. I do see good reason for non-Windows users to put up with some > inconvenience during the "beta" phase of implementing that solution; It's not that obvious -- we at least need the server-side check that doesn't allow "wrong" line endings as the "last" line of defense, and this check already needs a way to know which files are supposed to have which line endings -- deciding how to specify that is already half of the needed solution. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the charset issue
"Martin v. Löwis" wrote: >>>These files are in 8859-1 encoding (names in comments, at least): >>> http://svn.python.org/view/python/trunk/Lib/encodings/punycode.py >>> http://svn.python.org/view/python/trunk/Lib/test/test_csv.py >>> http://svn.python.org/view/python/trunk/Tools/i18n/msgfmt.py >>> http://svn.python.org/view/python/trunk/Tools/i18n/pygettext.py >>>If they are not marked as "text/plain; charset=iso-8859-1" I think it's >>> a bug. Either they should be marked, or converted to ascii or utf-8; the >>> coding pseudocomment (directive) should be changed accordingly. >> >> It's certainly ok to convert them to utf-8 (and add the marker anyway). > > No, it's not. PEP 8 mandates that non-ASCII code in the Python source > code is in Latin-1. Then I guess it's time to change PEP 8 for Python 2.7 ... """ Code in the core Python distribution should aways use the ASCII or UTF-8 encoding together with a PEP 263 encoding comment header. """ Since UTF-8 is ASCII compatible, the whole source code will effectively be UTF-8 encoded. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 05 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
> I'm not aware of any other unresolved items; they may exist, but the fact > that they're not discussed on this list in detail means that they are > largely unimportant. There is a long list of things that still need to be done; each one potentially creating new problems. In particular: - the .hgeols plugin needs to be written - the hooks need to be written, or at least deployed, for code style checks, for email notification, and for buildbot triggering - the build identification patch needs to be written (I do expect many problems out of that one, some possibly small - I'm not a Mercurial user, so I can't estimate how difficult that will be) - buildbot configuration needs to be adjusted - the roundup regex needs to be configured to refer to hgweb links - access control needs to be setup - stackless needs to be converted - a decision on the location of the PEPs must be made and implemented - developer documentation needs to be written - a decision must be made what to do with the migrated parts of subversion, in the subversion repository I may have missed some things. I would like to see test period (say, two weeks) were we can find further issues. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Martin v. Löwis schrieb: >> I'm not aware of any other unresolved items; they may exist, but the fact >> that they're not discussed on this list in detail means that they are >> largely unimportant. > > There is a long list of things that still need to be done; each one > potentially creating new problems. In particular: > - the .hgeols plugin needs to be written > - the hooks need to be written, or at least deployed, for code > style checks, for email notification, and for buildbot triggering > - the build identification patch needs to be written (I do expect > many problems out of that one, some possibly small - I'm not a > Mercurial user, so I can't estimate how difficult that will be) > - buildbot configuration needs to be adjusted > - the roundup regex needs to be configured to refer to hgweb links > - access control needs to be setup > - stackless needs to be converted > - a decision on the location of the PEPs must be made and implemented > - developer documentation needs to be written > - a decision must be made what to do with the migrated parts of > subversion, in the subversion repository > > I may have missed some things. I would like to see test period (say, > two weeks) were we can find further issues. Sure there are many things to do; I was speaking of issues where the way to go is not decided, and needs to be before the switch can happen. Maybe build identification is one of them; but I think everything has been said in the one thread we had about this. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: pruning/reorganizing branches
2009/8/3 Dirkjan Ochtman : > So PEP 385 proposes to clean up the old branches we still have lying > around in SVN. > > io-c: keep-clone? strip - it was merged into py3k some months ago. -- Amaury Forgeot d'Arc ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reasons for using expy
Yingjie Lan wrote: > Hi, > > The expy project provides an express way to extend Python. After some > careful considerations, I came up with some reasons for expy (this is > not an exhaustive list): This kind of advocacy for external projects belongs on python-list, not python-dev (or, if you're proposing something for use in the standard library, on python-ideas). Cheers, Nick. P.S. The message to capi-sig was probably on topic - certainly closer to being so than the inclusion of python-dev. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Glenn Linderman:
> and perhaps other things (and
> are there new Unicode control characters that could be used for line
> endings?),
Unicode includes Line Separator U+2028 and Paragraph Separator
U+2029 but they are rarely supported and very rarely used. They are a
pain to work with since they are 3 byte sequences in UTF-8. Visual
Studio does support them.
Python does not currently support these line separators such as in
this example which only reads 2 lines rather than 3:
with open("x.txt", "wb") as f:
f.write("a\nb\u2029c\n".encode('utf-8'))
with open("x.txt", "r") as f:
n = 1
for l in f.readlines():
print(n, repr(l))
n += 1
Neil
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reasons for using expy
> From: Nick Coghlan > Subject: Re: [Python-Dev] Reasons for using expy > To: "Yingjie Lan" > Cc: [email protected] > Date: Thursday, August 6, 2009, 1:44 AM > This kind of advocacy for external projects belongs on > python-list, not > python-dev (or, if you're proposing something for use in > the standard > library, on python-ideas). > Thanks Nick. Cheers, Yingjie ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
On 6/08/2009 12:28 AM, Stephen J. Turnbull wrote: Mark Hammond writes: > I'm not sure what point you are trying to make, but I believe it *is* > possible for a solution to be found here which will keep Windows users > happy. I'm guessing you haven't had much practical experience with this > problem, so probably don't see this is clearly as Windows users do. Mercurial is not only open source, it's written in Python. The problem is known to be hard in a practical sense, the existing solutions (written by non-Windows developers, of course) are judged to be insufficient by Windows users, and the non-Windows developers "probably don't see this is clearly as Windows users do." I think the implication is obvious. There will be no good solution until Windows users develop it. I don't see a good reason to wait for that. My conclusion is different. I'm not sure of the history of win32text, but it most certainly is now squarely in the hands of Windows users. Patches to win32text, or even general discussion is usually met with silence, and when prodded, the response is "sorry - we don't use that - it is a Windows problem." As a result, we end up in the position we are in now - win32text is great in theory but doesn't work in practice, attempts to make it work are met with indifference, and the "problem" stays squarely with Windows users. Non Windows users remain oblivious to the pain, Windows users stop bothering with the extension, and the repository post-commit hooks then cause different pain. Hence my conclusion that the answer is for any such support to be developed in conjunction with Windows users, but also in such a way that the solution works, almost identically, for non Windows users. By insisting all platforms eat the same dog-food, there is much more chance the glaringly obvious (to Windows users) issues are addressed. > I do see good reason for non-Windows users to put up with some inconvenience during the "beta" phase of implementing that solution; it's important enough to be fast-tracked, and doesn't need to be perfect for everybody to be tried (though it should not be allowed to endanger repo content, which seems unlikely but needs care since it's a potential disaster). And on the flip-side, I accept we may migrate without the agreed solution fully implemented - I'm happy to accept commitments about what *will* be done even if it isn't a reality for a short while... Cheers, Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] (try-except) conditional expression similar to (if-else) conditional (PEP 308)
I'm new to this list, so please excuse me if this topic has been
discussed, but I didn't
see anything similar in the archives.
I very often want something like a try-except conditional expression similar
to the if-else conditional.
An example of the proposed syntax might be:
x = float(string) except float('nan')
or possibly
x = float(string) except ValueError float('nan')
Here's a simple example: Converting a large list of strings to floats
where there may be errors
that I want returned as nan's.
Currently I would write the function:
def safe_float_function(string):
try:
result = float(string)
except:
result = float('nan')
return result
and get my list of floats using the list comprehension:
xs = [ safe_float_function(string) for string in strings ]
With a try-except conditional I would instead define the following lambda:
safe_float_conditional = lambda string : float(string) except
float('nan')
leading to:
xs = [ safe_float_conditional(string) for string in strings ]
My understanding is that the second would be faster at run time, and,
like if-else conditional expressions,
possibly more easily read by the human.
Again, please excuse me if this has been discussed previously. If so,
I'd appreciate being pointed to the discussion.
Please also excuse me if for there is some currently (pre-python 3.0)
idiom that I could use to efficiently get this
same behaviour. If so, I'd appreciate being educated.
Thanks,
Jeff McAninch
--
==
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: [email protected]
==
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (try-except) conditional expression similar to (if-else) conditional (PEP 308)
[Jeffrey E. McAninch, PhD]
I very often want something like a try-except conditional expression similar
to the if-else conditional.
An example of the proposed syntax might be:
x = float(string) except float('nan')
or possibly
x = float(string) except ValueError float('nan')
+1
I've long wanted something like this.
One possible spelling is:
x = float(string) except ValueError else float('nan')
If accepted, this would also solve the feature requests for various functions
to have default arguments.
For example:
x = min(seq) except ValueError else 0 # default to zero for empty
sequences
It would also be helpful in calculations that have algebraic restrictions:
sample_std_deviation = sqrt(sum(x - mu for x in seq) / (len(seq)-1)) except
ZeroDivisionError else float('Inf')
Raymond
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (try-except) conditional expression similar to (if-else) conditional (PEP 308)
At 05:59 PM 8/5/2009 -0700, Raymond Hettinger wrote:
[Jeffrey E. McAninch, PhD]
I very often want something like a try-except conditional expression similar
to the if-else conditional.
An example of the proposed syntax might be:
x = float(string) except float('nan')
or possibly
x = float(string) except ValueError float('nan')
+1 I've long wanted something like this.
One possible spelling is:
x = float(string) except ValueError else float('nan')
I think 'as' would be better than 'else', since 'else' has a
different meaning in try/except statements, e.g.:
x = float(string) except ValueError, TypeError as float('nan')
Of course, this is a different meaning of 'as', too, but it's not
"as" contradictory, IMO... ;-)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (try-except) conditional expression similar to (if-else) conditional (PEP 308)
Raymond Hettinger wrote:
If accepted, this would also solve the feature requests for various
functions to have default arguments.
For example:
x = min(seq) except ValueError else 0 # default to zero for
empty sequences
It would also be helpful in calculations that have algebraic
restrictions:
sample_std_deviation = sqrt(sum(x - mu for x in seq) / (len(seq)-1))
except ZeroDivisionError else float('Inf')
Raymond
Yes, exactly the situations I keep coding around.
--
==
Jeffrey E. McAninch, PhD
Physicist, X-2-IFD
Los Alamos National Laboratory
Phone: 505-667-0374
Email: [email protected]
==
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
"Martin v. Löwis" writes: > > I don't see any good reason for having any encodings but UTF-8 in > > Python's. > > Just in case my previous message gets overlooked: PEP 8 mandates Latin-1 > for Python 2.x source code (except for files that test PEP 263). You're right, sorry for the misinformation. An exception should be made for gettext message files, too? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: Mercurial issues
> > Just in case my previous message gets overlooked: PEP 8 mandates Latin-1 > > for Python 2.x source code (except for files that test PEP 263). > > You're right, sorry for the misinformation. > > An exception should be made for gettext message files, too? In principle, perhaps. However, Python doesn't have any .po files, AFAIK. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
Mark Hammond writes: > On 6/08/2009 12:28 AM, Stephen J. Turnbull wrote: > > I think the implication is obvious. There will be no good solution > > until Windows users develop it. I don't see a good reason to wait for > > that. > My conclusion is different. I'm not sure of the history of win32text, > but it most certainly is now squarely in the hands of Windows users. > Patches to win32text, or even general discussion is usually met with > silence, and when prodded, the response is "sorry - we don't use that - > it is a Windows problem." Well, yes, it is a Windows problem. And it will probably always be that way, because for practical purposes, Windows users cannot advocate their platform's infrastructure solutions for open source projects: those solutions are proprietary. On the flip side, in my experience at least Windows users do not contribute much to this kind of infrastructure initiative, undoubtedly due to the high cost of acquiring familiarity with the usable options[1], and so have less input into the process. But that's a matter of certain costs that are built in to the nature of a proprietary platform. Somebody has to pay them, and I think it should be the users of that platform. Why should the rest of the community subsidize that platform? > As a result, we end up in the position we are in now - win32text is > great in theory but doesn't work in practice, attempts to make it work > are met with indifference, and the "problem" stays squarely with Windows > users. This is simply false AFAICS. There was little participation on this particular issue during PEP 374 that I can recall. Now that it is clearly an issue after all, it's still early in the PEP 385 process. Martin has already picked up the ball on EOL support, and has carried informal design pretty much to the goal line already ... all that's left is the detailed design and the implementation, and there are several people involved who will help develop the patch, all very capable. (Of course it's going to be easier said than done and there are probably bumps in the road to a smooth workflow, but I do claim that the process is working as well as you could expect.) > Hence my conclusion that the answer is for any such support to be > developed in conjunction with Windows users, [...] Ahem. Why not "(primarily) by Windows users"? > And on the flip-side, I accept we may migrate without the agreed > solution fully implemented - I'm happy to accept commitments about > what *will* be done even if it isn't a reality for a short while... Make no mistake about it, EOL support is a tempest in a teapot compared to the benefits to a large number of core developers in their *personal* workspaces -- even if the project workflow doesn't change at all. That's what is driving this change. Unless Windows users do it themselves, they are dependent on the good will of the PEP 385 proponent and other volunteer contributors. I don't think "accepting commitments" is part of the game plan. Footnotes: [1] Eg, I was willing to participate in PEP 374 because I already have a great interest in version control and use git daily. Lots of Unix users don't, and they didn't participate any more than most Windows users did. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: the eol-type issue
> This is simply false AFAICS. There was little participation on this > particular issue during PEP 374 that I can recall. Now that it is > clearly an issue after all, it's still early in the PEP 385 process. > Martin has already picked up the ball on EOL support, and has carried > informal design pretty much to the goal line already ... all that's > left is the detailed design and the implementation, and there are > several people involved who will help develop the patch, all very > capable. I'm not so optimistic. To me, it looks like that either Dirkjan or Mark will implement a hg hook, or else it won't happen (for me, I certainly know that I will not write Mercurial hooks anytime soon). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
