[Python-Dev] PEP 385: updating the PEP
The diff below should reflect changes from the discussion we had last time. Please review. (Some comments may be more appropriate for the other threads I just kicked off.) Cheers, Dirkjan Index: pep-0385.txt === --- pep-0385.txt(revision 74294) +++ pep-0385.txt(revision 74296) @@ -59,27 +59,25 @@ often has somewhat unintuitive results for people (though this has been getting better in recent versions of Mercurial). -I'm still a bit on the fence about whether Python should adopt cloned -branches and named branches. Since it usually makes more sense to tag releases -on the maintenance branch, for example, mainline history would not contain -release tags if we used cloned branches. Also, Mercurial 1.2 and 1.3 have the -necessary tools to make named branches less painful (because they can be -properly closed and closed heads are no longer considered in relevant cases). +The current proposal is to use named branches for release branches and adopt +cloned branches for feature branches, with one exception to this rule: the 3.x +branches will be kept in separate clones from the 2.x branches. I think this +provides an optimal hybrid approach for Python's uses of branching. -A disadvantage might be that the used clones will be a good bit larger (since -they essentially contain all other branches as well). This can me mitigated by -keeping non-release (feature) branches in separate clones. Also note that it's -still possible to clone a single named branch from a combined clone, by -specifying the branch as in hg clone http://hg.python.org/main/#2.6-maint. -Keeping the py3k history in a separate clone problably also makes sense. +Differences between named branches and cloned branches: -XXX To do: size comparison for selected separation scenarios. +* Tags in a different (maintenance) clone aren't available in the local clone +* Clones with named branches will be larger, since they contain more data +(The Mercurial book discourages the use of named branches, but it is, in this +respect, somewhat outdated. Named branches have gotten much easier to use +since that comment was written, due to improvements in hg.) + Converting branches --- There are quite a lot of branches in SVN's branches directory. I propose to -clean this up a bit, by employing the following the strategy: +clean this up a bit, by following this basic strategy: * Keep all release (maintenance) branches * Discard branches that haven't been touched in 18 months, unless somone @@ -87,6 +85,21 @@ * Keep branches that have been touched in the last 18 months, unless someone indicates the branch can be deprecated +There's a `branch map`_ available that shows info about each branch: + +* keep-clone means we'll keep that branch in a separate clone +* keep-named means we'll keep that branch as a named branch in one of the clones +* strip means we won't keep that branch +* streamed-merge means that it got merged by committing several new revisions + to the other branch +* merged-r* means the branch got merged in the named revision +* merges? means I haven't checked/found out yet whether that branch was ever + merged +* ? means that your input would be even more helpful than for the other items +* some items have no action yet, feel free to treat that as just '?' + +.. _branch map: http://hg.python.org/pymigr/file/tip/all-branches.txt + Converting tags --- @@ -95,8 +108,8 @@ we should keep all release tags, and consider other tags for inclusion based on requests from the developer community. I'd like to consider unifying the release tag naming scheme to make some things more consistent, if people feel -that won't create too many problems. For example, Mercurial itself just uses -'1.2.1' as a tag, where CPython would currently use r121. +that won't create too many problems. The current proposal is to bring old +release tags in line with the current practice of release tag naming. Author map -- @@ -119,17 +132,19 @@ possible forms of pattern matching. The current Python repository already includes a rudimentary .hgignore file to help with using the hg mirrors. -It might be useful to have the .hgignore be generated automatically from -svn:ignore properties. This would make sure all historic revisions also have -useful ignore information (though one could argue ignoring isn't really -relevant to just checking out an old revision). +Since the current Python repository already includes a .hgignore file (for use +with hg mirrors), we'll just use that. Generating full history of the file +was debated but deemed impractical (because it's relatively hard with fairly +little gain, since ignoring is less important for older revisions). Revlog reordering - -As an optional optimization technique, we should consider trying a reordering -pass on the revlogs (internal Mercurial files) resulting from the conversion.
[Python-Dev] PEP 385: the eol-type issue
So, I've been not-working on this, which I feel bad about. Suffice it to say the day job has required more of my time then usual for the past few weeks. I want to get back into it, so let's start by re-raising this issue, which Mark Hammond conveniently summarized below. > On 4/07/2009 2:03 PM, Mark Hammond wrote: >> On 4/07/2009 12:30 PM, Nick Coghlan wrote: >>> And since Mercurial doesn't even allow us to say "this is a binary file" >>> the way CVS used to I'm currently not seeing any way for that to happen >>> except for win32text to be updated to correctly handle wild cards in >>> combination with negative filters. >> >> I agree with your conclusion. My ruminating on this over the last few >> months leaves me thinking this would involve: >> >> * my older 'accepted but then lost' hg patch to allow an explicit 'none' >> rule for a single file to override wildcards. This was and still is a good idea. It would be very nice if you could un-bitrot it and submit it for inclusion into crew-stable (so that it may land in the next release, which would hopefully be a somewhat near 1.3.2). >> * win32text be enhanced to use a normal versioned file in the root of >> the repo, much like hgingore, where a project can maintain project wide >> rules. I'm thinking that it should take stuff from .hgeols or whatever and apply rules from .hg/hgrc after that, so both may be used (and for backwards compatibility), but it sounds like a good idea in principle. >> * win32text be enhanced such that all python developers, regardless of >> platform, are willing to use this extension, even if the majority of >> files happen to use their native line ending (sauce for the goose is >> sauce for the gander, and all that...) I don't think that is necessary, I will elaborate below. >> * commit hooks be implemented to enforce this - but this should not be >> necessary if the above was implemented and socially enforced. You seem to advocate a two-step approach: enforce line endings through win32text, catch any errors that slipped through in a hook (commit hook is an optional first line of defense, changegroup hooks on the server to protect the rest of the world). I think inverting that approach would be better: have strict hooks on the server to prevent people from pushing inappropriate EOLs, and provide help on configuring win32text as an extra help for developers on Windows who use editors that work better with \r\n. That leaves people to pick their own weapon of choice against propagation of \r\n (e.g. better editor, commit hooks, whatever) while still making sure no inappropriate line endings land in the python.org repositories. It also seems to fit well with the whole consenting adults thing (but that might just be me). On Sun, Jul 19, 2009 at 15:27, Mark Hammond wrote: > Sorry Dirkjan - I just noticed I didn't CC you on this mail originally. I'm > wondering if you have any more thoughts on these EOL issues and if there is > anything I can do to help? Taking up the 'none' filter, first, and .hgeols, secondly, in the win32text extension would be wonderfully helpful, since I don't do much development on Windows and am therefore not that familiar with the extension in the first place. Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: updating the PEP
Hello Dirkjan, Dirkjan Ochtman ochtman.nl> writes: > > +As an optional optimization technique, I have performed a reordering pass on > +the revlogs (internal Mercurial files) resulting from the conversion. In some > +cases this results in dramatic decreases in on-disk repository size. Can you give size numbers for the two main repositories (2.x and 3.x)? Thanks for your work, again! I'm glad this is progressing. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] In late this am
Going to the Dr., will be in thereafter. -Casey ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] In late this am
Heh, wrong dev list 8^). Sorry for the noise. -Casey On Aug 3, 2009, at 8:47 AM, Casey Duncan wrote: Going to the Dr., will be in thereafter. -Casey ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pylinting the stdlib
On Sun, Aug 2, 2009 at 6:20 PM, Mark Dickinson wrote: > On Sat, Aug 1, 2009 at 11:40 PM, Vincent Legoll > wrote: >> >> I've fed parts of the stdlib to pylint and after some filtering >> there appears to be some things that looks strange, I've >> filled a few bugs to the tracker for them. >> >> Is this useless and taking reviewer's time for nothing ? >> >> Please advise, if this is deemed useful, I'll continue further > > I think this is valuable work---please do continue! Thanks, I will > Just out of interest, how many false positives did you have > to filter out in finding the 5 cases above? I can't really tell if there was false positives, I just started with the low hanging fruits, the ones I immediately saw as fishy, the remaining I skipped without too much consideration, I think it will take many iterations to do the whole thing. I used a pylint version which is not capable of understanding py3k syntax, so a lot of files were simply skipped. -- Vincent Legoll ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Functionality in subprocess.Popen.terminate()
In my GSoC project, I have implemented asnychronous I/O in subprocess.Popen. Since the read/write operations are asynchronous, the program may have already exited by the time one calls the asyncread function I have implemented. While it returns the data just fine, I have come across an issue with the TerminateProcess function in Windows: if the program has already exited, when subprocess.Popen.Terminate calls the Windows built-in "TerminateProcess" function, an "access denied" error will occur. Should I just make it so that this exception is simply ignored or perform some kind of check to see if the process exists beforehand? If the latter, I have been unable to find a way to do so, to my liking at least. The solutions I saw would require code that seems a bit excessive to me. Eric ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: updating the PEP
On Mon, Aug 3, 2009 at 04:53, Dirkjan Ochtman wrote: > The diff below should reflect changes from the discussion we had last > time. Please review. (Some comments may be more appropriate for the > other threads I just kicked off.) > > Cheers, > > Dirkjan > > Index: pep-0385.txt > === > --- pep-0385.txt(revision 74294) > +++ pep-0385.txt(revision 74296) > @@ -59,27 +59,25 @@ > often has somewhat unintuitive results for people (though this has been > getting better in recent versions of Mercurial). > > -I'm still a bit on the fence about whether Python should adopt cloned > -branches and named branches. Since it usually makes more sense to tag > releases > -on the maintenance branch, for example, mainline history would not contain > -release tags if we used cloned branches. Also, Mercurial 1.2 and 1.3 have > the > -necessary tools to make named branches less painful (because they can be > -properly closed and closed heads are no longer considered in relevant > cases). > +The current proposal is to use named branches for release branches and > adopt > +cloned branches for feature branches, with one exception to this rule: the > 3.x > +branches will be kept in separate clones from the 2.x branches. I think > this > +provides an optimal hybrid approach for Python's uses of branching. > Sounds good to me. > > -A disadvantage might be that the used clones will be a good bit larger > (since > -they essentially contain all other branches as well). This can me > mitigated by > -keeping non-release (feature) branches in separate clones. Also note that > it's > -still possible to clone a single named branch from a combined clone, by > -specifying the branch as in hg clone > http://hg.python.org/main/#2.6-maint. > -Keeping the py3k history in a separate clone problably also makes sense. > +Differences between named branches and cloned branches: > > -XXX To do: size comparison for selected separation scenarios. > +* Tags in a different (maintenance) clone aren't available in the local > clone > +* Clones with named branches will be larger, since they contain more data > > +(The Mercurial book discourages the use of named branches, but it is, in > this > +respect, somewhat outdated. Named branches have gotten much easier to use > +since that comment was written, due to improvements in hg.) > + > Converting branches > --- > > There are quite a lot of branches in SVN's branches directory. I propose > to > -clean this up a bit, by employing the following the strategy: > +clean this up a bit, by following this basic strategy: > > * Keep all release (maintenance) branches > * Discard branches that haven't been touched in 18 months, unless somone > @@ -87,6 +85,21 @@ > * Keep branches that have been touched in the last 18 months, unless > someone > indicates the branch can be deprecated > > +There's a `branch map`_ available that shows info about each branch: > + > +* keep-clone means we'll keep that branch in a separate clone > +* keep-named means we'll keep that branch as a named branch in one of > the clones > +* strip means we won't keep that branch > +* streamed-merge means that it got merged by committing several new > revisions > + to the other branch > +* merged-r* means the branch got merged in the named revision > +* merges? means I haven't checked/found out yet whether that branch was > ever > + merged > +* ? means that your input would be even more helpful than for the other > items > +* some items have no action yet, feel free to treat that as just '?' > + > +.. _branch map: http://hg.python.org/pymigr/file/tip/all-branches.txt > + > Converting tags > --- > > @@ -95,8 +108,8 @@ > we should keep all release tags, and consider other tags for inclusion > based > on requests from the developer community. I'd like to consider unifying > the > release tag naming scheme to make some things more consistent, if people > feel > -that won't create too many problems. For example, Mercurial itself just > uses > -'1.2.1' as a tag, where CPython would currently use r121. > +that won't create too many problems. The current proposal is to bring old > +release tags in line with the current practice of release tag naming. > > Author map > -- > @@ -119,17 +132,19 @@ > possible forms of pattern matching. The current Python repository already > includes a rudimentary .hgignore file to help with using the hg mirrors. > > -It might be useful to have the .hgignore be generated automatically from > -svn:ignore properties. This would make sure all historic revisions also > have > -useful ignore information (though one could argue ignoring isn't really > -relevant to just checking out an old revision). > +Since the current Python repository already includes a .hgignore file (for > use > +with hg mirrors), we'll just use that. Generating full history of the file > +was debated but deemed impractical (be
[Python-Dev] PEP 385: pruning/reorganizing branches
So PEP 385 proposes to clean up the old branches we still have lying around in SVN. This list of branch: action items is what I've come up with to do this cleanup. Legend first: - keep-clone means we'll keep that branch in a separate clone - keep-named means we'll keep that branch as a named branch in one of the clones - strip means we won't keep that branch - streamed-merge means that it got merged by committing several new revisions to the other branch - merged-r* means the branch got merged in the named revision - merges? means I haven't checked/found out yet whether that branch was ever merged - ? means that your input would be even more helpful than for the other items - some items have no action yet, feel free to treat that as just '?' The actual List: py3k: keep-clone default: keep-clone tk_and_idle_maintenance: keep-clone release26-maint: keep-named release30-maint: keep-named pep-0383: keep-clone py3k-short-float-repr: strip streamed-merge multiprocessing-autoconf: keep-clone? release25-maint: keep-named io-c: keep-clone? py3k-issue1717: keep-clone tlee-ast-optimize: keep-clone release24-maint: keep-named empty: keep-clone? py3k-urllib: keep-clone tnelson-trunk-bsddb-47-upgrade: strip benjaminp-testing: strip py3k-importlib: keep-clone release23-maint: keep-named py3k-importhook: keep-clone ctypes-branch: strip decimal-branch: merged-r58143 bcannon-objcap: keep-clone? p3yk_no_args_on_exc: strip amk-mailbox: keep-clone? twouters-dictviews-backport: keep-clone? bcannon-sandboxing: keep-clone? release22-maint: keep-named theller_modulefinder: strip hoxworth-stdlib_logging-soc: strip partial tim-exc_sanity: merged-r46426 IDLE-syntax-branch: merged-r41480 strip-later ast-objects: strip ast-arena: merged-r41739 jim-doctest: strip ast-branch: merged-r39758 release23-branch: merges? jim-modulator: strip release21-maint: keep-named indexing-cleanup-branch: strip r23c1-branch: merged-r33637 r23b2-branch: merges? anthony-parser-branch: merged-r35460 r23b1-branch: merged-r32490 idlefork-merge-branch: strip getargs_mask_mods: strip cache-attr-branch: strip folding-reimpl-branch: strip streamed-merge r23a2-branch: merges? bsddb-bsddb3-schizo-branch: merged-r31008 r23a1-branch: merged-r30482 py-cvs-vendor-branch: strip DS_RPC_BRANCH: strip streamed-merge SourceForge: strip release22-branch: merged-r24921 r22rc1-branch: strip r22b2-branch: merges? merged-r24426 r22b1-branch: merges? r22a4-branch: merges? r22a3-branch: merges? r22a2-branch: merged-r22674 descr-branch: merged-r22139 release20-maint: keep-named gen-branch: merged-r21181 iter-branch: merged-r20492 r161-branch: merges? cnri-16-start: strip universal-33: merges? None: strip avendor: strip Distutils_0_1_3-branch: strip partial release152p1-patches: merges? string_methods: merged-r13927 PYIDE: strip OSAM: strip PYTHONSCRIPT: strip BBPY: strip jar: merges? alpha100: strip streamed-merge unlabeled-2.36.4: strip partial unlabeled-2.1.4: strip partial unlabeled-2.25.4: strip partial fix-test-ftplib: merged-r66673 trunk-math: py3k-grandrenaming: okkoto-sizeof py3k-ctypes-pep3118: merged-r62597 trunk-bytearray: merged-r61936 libffi3-branch: merged-r61234 alex-py3k: strip cpy_merge: strip py3k-pep3137: merged-r5 ../ctypes-branch: strip pep302_phase2: strip py3k-buffer: merged-r57181 p3yk-noslice py3k-struni p3yk: rename int_unification unlabeled-1.1.1: strip unlabeled-1.5.4: strip unlabeled-1.1.2: strip unlabeled-2.9.2: strip unlabeled-2.9.4: strip unlabeled-1.5.2: strip unlabeled-2.1.2: strip unlabeled-2.36.2: strip unlabeled-2.108.2: strip unlabeled-2.10.2: strip unlabeled-2.54.2: strip unlabeled-1.3.2: strip unlabeled-1.23.4: strip unlabeled-2.25.2: strip unlabeled-1.2.2: strip unlabeled-1.98.2: strip unlabeled-2.16.2: strip unlabeled-2.3.2: strip unlabeled-1.9.2: strip unlabeled-1.8.2: strip aimacintyre-sf1454481: merged-r46919 tim-current_frames: merged-r50541 bippolito-newstruct: merges? runar-longslice-branch: strip steve-notracing: strip rjones-funccall: merged-r46096 sreifschneider-newnewexcept: merged-r46456 tim-doctest-branch: merged-r36839 blais-bytebuf: strip ../bippolito-newstruct: rename rjones-prealloc: strip sreifschneider-64ints: strip stdlib-cleanup: strip ssize_t: merged-r42382 sqlite-integration: merged-r43514 tim-obmalloc: merged-r43059 Further actions: - implement branch map support in hgsubversion to be able to do named/unnamed/no branch on a branch-by-branch basis - implement splice map support in hgsubversion to be able to convert given merges to hg-native merge data Cheers, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: pruning/reorganizing branches
> empty: keep-clone? I use that as a branch to tell build slaves to clean out their current checkouts. So keep-clone sounds right, assuming it is possible to target buildslaves at either clones or branches (which, IIUC, would be necessary anyway, since we are using a mix of branches and clones). > amk-mailbox: keep-clone? > twouters-dictviews-backport: keep-clone? > bcannon-sandboxing: keep-clone? > bippolito-newstruct: merges? You'll probably need to explicitly ping the specific owners (Andrew Kuchling, Thomas Wouters, Brett Cannon, Bob Ippolito) to understand the fate of these branches. This also raises the question how developers should publish their "own" branches. For the bzr setup, there was apparently a proposal to use directories for that, i.e. giving each developer a directory on code.python.org to publish branches. Not doing that, but keeping owner information encoded in the clone name, would be fine as well. > release23-branch: merges? > r23b2-branch: merges? > r22rc1-branch: strip > r22b1-branch: merges? > r22a4-branch: merges? > r22a3-branch: merges? > r161-branch: merges? It seems we had been creating CVS branches for every release around that time; I don't remember the details. Each such branch should end up in a tag. For example, release23-branch should (and does) ultimately lead to tags/r23. cvs2svn wasn't able to recognize this correctly (as CVS branches apply to each file individually), so it created the r23 tag out of various copies that were current when the tag was made. I don't know what your plan is wrt. release tags, i.e. whether you want to keep them all. If you are stripping out some of the branches, but plan to keep the release tags, I wonder what the tags look like. > release22-branch: merged-r24921 Not really. Jack Jansen merged some changes that got first applied to the 2.2 > r22b2-branch: merges? merged-r24426 > r22b2-branch: merges? merged-r24426 > release20-maint: keep-named See above. So you do plan to keep all past releases? > release152p1-patches: merges? Probably merged. I don't recall whether 1.5.2p1 really happened; in r14966, Fred claims that he merged all changes from 1.5.2p2 (!). "Hopefully I got all this right!" I surely hope the same - I doubt anybody would go back and check whether anything is missing. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 385: pruning/reorganizing branches
On Tue, Aug 4, 2009 at 08:06, "Martin v. Löwis" wrote: > I use that as a branch to tell build slaves to clean out their > current checkouts. So keep-clone sounds right, assuming it is possible > to target buildslaves at either clones or branches (which, IIUC, would > be necessary anyway, since we are using a mix of branches and clones). Yes, that should be straightforward. >> amk-mailbox: keep-clone? >> twouters-dictviews-backport: keep-clone? >> bcannon-sandboxing: keep-clone? >> bippolito-newstruct: merges? > > You'll probably need to explicitly ping the specific owners > (Andrew Kuchling, Thomas Wouters, Brett Cannon, Bob Ippolito) > to understand the fate of these branches. Will do. > This also raises the question how developers should publish their > "own" branches. For the bzr setup, there was apparently a proposal > to use directories for that, i.e. giving each developer a directory > on code.python.org to publish branches. User repositories has apparently worked well for Mozilla, so yeah, it's worth discussing. > Not doing that, but keeping owner information encoded in the clone > name, would be fine as well. > >> release23-branch: merges? >> r23b2-branch: merges? >> r22rc1-branch: strip >> r22b1-branch: merges? >> r22a4-branch: merges? >> r22a3-branch: merges? >> r161-branch: merges? > > It seems we had been creating CVS branches for every release around > that time; I don't remember the details. Each such branch should end > up in a tag. For example, release23-branch should (and does) ultimately > lead to tags/r23. cvs2svn wasn't able to recognize this correctly (as > CVS branches apply to each file individually), so it created the r23 > tag out of various copies that were current when the tag was made. > > I don't know what your plan is wrt. release tags, i.e. whether you > want to keep them all. If you are stripping out some of the branches, > but plan to keep the release tags, I wonder what the tags look like. The plan was to keep all maintenance branches and all release tags but not all release branches (since they seem to contain few commits anyway). >> release22-branch: merged-r24921 > > Not really. Jack Jansen merged some changes that got first applied > to the 2.2 > >> r22b2-branch: merges? merged-r24426 >> r22b2-branch: merges? merged-r24426 > >> release20-maint: keep-named > > See above. So you do plan to keep all past releases? > >> release152p1-patches: merges? > > Probably merged. I don't recall whether 1.5.2p1 really happened; > in r14966, Fred claims that he merged all changes from 1.5.2p2 (!). > > "Hopefully I got all this right!" > > I surely hope the same - I doubt anybody would go back and check > whether anything is missing. Thanks for the thorough review, Dirkjan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
