Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-26 Thread Victor Stinner 
Hi,

I updated my bisect_test.py script to simplify its command line:

   https://github.com/haypo/misc/blob/master/python/bisect_test.py

To debug a reference leak in test_os, now just type:

   $ ./python bisect_test.py -R 3:3 test_os

Example of output:
---
(...)
Failing tests (1):
* test.test_os.ExecTests.test_execve_invalid_env

Bisection completed in 24 iterations and 0:02:11
---

In 2 minutes, you get the guilty: test_execve_invalid_env()! It's now
fully automated!


I also added --fail-env-changed option to regrtest. Using this option,
you can find which test leaked a file, and modified any other resource
tracked by regrtest. For examle, I used this command to find which
test method of test_subprocess created a core dump file:

   $ ./python bisect_test.py --fail-env-changed test_subprocess


Later, I will try to integrate this tiny tool in Python. Maybe
somewhere in Tools/. Or better, maybe directly in regrtest?

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New workflow change: Welcome to blurb

2017-06-26 Thread Brett Cannon 
On Sun, 25 Jun 2017 at 20:56 Nick Coghlan  wrote:

> On 26 June 2017 at 02:39, Brett Cannon  wrote:
> > On Sat, Jun 24, 2017, 23:45 Nick Coghlan,  wrote:
> >> I was mainly referring to cases where either an initial commit (with a
> >> complete NEWS entry) requires follow-up fixes (no NEWS edit
> >> necessary).
> >>
> >> However, another case did occur to me, which is fixing bugs reported
> >> against never-released features (which is what I'm currently working
> >> on for PEP 538).
> >
> > https://github.com/python/bedevere/pull/22 is the current
> implementation of
> > the check. All it does is see if the PR has a path that contains a news
> > file. It can also easily be changed to also support some "no news" label
> if
> > people want it to.
>
> Yeah, I think I'd prefer to see the current "trivial" label split into
> the more explicit "no issue needed" and "no NEWS needed". It isn't
> necessarily the case that the changes in such PRs are trivial, it's
> that the usual checks don't apply for some reason (and the change
> being genuinely trivial is only one possibility)
>

I've opened https://github.com/python/bedevere/issues/25 to change this for
the issue number check (although I will name the label "skipped issue" so
all check-skipping labels and start with the same prefix for sorting
purpose).
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-26 Thread Brett Cannon 
On Mon, 26 Jun 2017 at 09:38 Victor Stinner 
wrote:

> Hi,
>
> I updated my bisect_test.py script to simplify its command line:
>
>https://github.com/haypo/misc/blob/master/python/bisect_test.py
>
> To debug a reference leak in test_os, now just type:
>
>$ ./python bisect_test.py -R 3:3 test_os
>
> Example of output:
> ---
> (...)
> Failing tests (1):
> * test.test_os.ExecTests.test_execve_invalid_env
>
> Bisection completed in 24 iterations and 0:02:11
> ---
>
> In 2 minutes, you get the guilty: test_execve_invalid_env()! It's now
> fully automated!
>
>
> I also added --fail-env-changed option to regrtest. Using this option,
> you can find which test leaked a file, and modified any other resource
> tracked by regrtest. For examle, I used this command to find which
> test method of test_subprocess created a core dump file:
>
>$ ./python bisect_test.py --fail-env-changed test_subprocess
>
>
> Later, I will try to integrate this tiny tool in Python. Maybe
> somewhere in Tools/. Or better, maybe directly in regrtest?
>

I don't see why regrtest isn't the right place for this.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-26 Thread Victor Stinner 
2017-06-26 21:58 GMT+02:00 Brett Cannon :
> I don't see why regrtest isn't the right place for this.

The current regrest CLI isn't designed for subcommands, and I don't
want to "pollute" regrtest with multiple options for bisect.
Currently, my script has already 4 options:

haypo@selma$ python3 ~/prog/GIT/misc/python/bisect_test.py --help
usage: bisect_test.py [-h] [-i INPUT] [-o OUTPUT] [-n MAX_TESTS] [-N MAX_ITER]

optional arguments:
  -h, --helpshow this help message and exit
  -i INPUT, --input INPUT
Test names produced by --list-tests written into a
file. If not set, run --list-tests
  -o OUTPUT, --output OUTPUT
Result of the bisection
  -n MAX_TESTS, --max-tests MAX_TESTS
Maximum number of tests to stop the bisection
(default: 1)
  -N MAX_ITER, --max-iter MAX_ITER
Maximum number of bisection iterations (default: 100)

I really like subcommands, it's a nice way to design complex CLI ;-)

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New workflow change: Welcome to blurb

2017-06-26 Thread Terry Reedy 

On 6/25/2017 4:09 PM, Guido van Rossum wrote:
On Sun, Jun 25, 2017 at 12:23 PM, Brett Cannon > wrote:




On Sun, Jun 25, 2017, 11:24 Terry Reedy, mailto:[email protected]>> wrote:




I was not aware that PRs could be edited via a web UI and have
no idea
how.  I search a PR for something, searched the devguide index for
'web', and searched 3 Lifecycle of a pull request.  Did I miss
something.  Also, 6. Helping with Documentation should say something
about submitting typo PRs via the web, if indeed that is
possible yet.


You can edit anything via the web, it's just part of GitHub and so
we have not documented it explicitly to avoid just duplicating
GitHub's own docs.


There is a different between 'duplicating' the docs and providing short 
pointers.


If you're still not sure how, from the PR UI go to the "Files changed" 
tab and click the "pencil" icon on the heading for the file you want to 
change. The PR creator must have enabled this (but I think it's on by 
default). When you save, GitHub creates a new commit in the PR's branch 
and all the usual tests (Travis-CI etc.) get kicked off. Where it breaks 
down is if you want to edit multiple files (it creates a new commit for 
each) or if the files are large (scrolling around is awkward) or if you 
want to make extensive changes (the web editor is limited in its 
capabilities, it's based on https://codemirror.net/).


Thank you.  This will make reviewing more fun. This should speed up 
editing doc strings and comments and making other small changes (in a 
single file) while reviewing.  I just used this to edit a news item I 
pushed previously.


It seems strange that such an edit triggers a travis download of 3.6 to 
run the cherry_picker test, but that is another issue.


Maybe I will use the web editor to add a a couple of lines to the 
devguide about using the web editor ;-).


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New workflow change: Welcome to blurb

2017-06-26 Thread Brett Cannon 
On Mon, 26 Jun 2017 at 14:28 Terry Reedy  wrote:

> On 6/25/2017 4:09 PM, Guido van Rossum wrote:
> > On Sun, Jun 25, 2017 at 12:23 PM, Brett Cannon  > > wrote:
> >
> >
> >
> > On Sun, Jun 25, 2017, 11:24 Terry Reedy,  > > wrote:
> >
>
> > I was not aware that PRs could be edited via a web UI and have
> > no idea
> > how.  I search a PR for something, searched the devguide index
> for
> > 'web', and searched 3 Lifecycle of a pull request.  Did I miss
> > something.  Also, 6. Helping with Documentation should say
> something
> > about submitting typo PRs via the web, if indeed that is
> > possible yet.
> >
> >
> > You can edit anything via the web, it's just part of GitHub and so
> > we have not documented it explicitly to avoid just duplicating
> > GitHub's own docs.
>
> There is a different between 'duplicating' the docs and providing short
> pointers.
>

Yes, but I've been using GitHub for years so I don't know what is obvious
and isn't anymore. And there is also a balance when documenting things that
we don't control and thus can go stale in the devguide.

This is the sort of thing we could point out in a "pointers on GitHub" page
once someone has time to re-organize the devguide around use-cases (e.g.
external contributor, core dev who doesn't use GitHub day-to-day, and How
we differ from other GitHub projects).


>
> > If you're still not sure how, from the PR UI go to the "Files changed"
> > tab and click the "pencil" icon on the heading for the file you want to
> > change. The PR creator must have enabled this (but I think it's on by
> > default). When you save, GitHub creates a new commit in the PR's branch
> > and all the usual tests (Travis-CI etc.) get kicked off. Where it breaks
> > down is if you want to edit multiple files (it creates a new commit for
> > each) or if the files are large (scrolling around is awkward) or if you
> > want to make extensive changes (the web editor is limited in its
> > capabilities, it's based on https://codemirror.net/).
>
> Thank you.  This will make reviewing more fun. This should speed up
> editing doc strings and comments and making other small changes (in a
> single file) while reviewing.  I just used this to edit a news item I
> pushed previously.
>
> It seems strange that such an edit triggers a travis download of 3.6 to
> run the cherry_picker test, but that is another issue.
>

Why is it strange? You changed something in the repository and Travis is
there to test that repository (and Travis doesn't know what you do and do
not want tested). And Python 3.6 doesn't come in the version of Ubuntu they
use so they have to get it somehow (and by downloading they can do stuff
like run release candidates or let people run against development versions
of Python).


>
> Maybe I will use the web editor to add a a couple of lines to the
> devguide about using the web editor ;-).
>

:) Seems appropriate if there's a place to put that ATM.

-Brett


>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Helping contributors with chores (do we have to?)

2017-06-26 Thread Brett Cannon 
[It seems to me most of this thread has gone off-topic since it now
includes what I consider a sexist comment, complaining about git, and
discussing people's credentials in terms of being able to comment on
something, I'm basically ignoring everything that came after this email as
none of that has anything to do with what Antoine is complaining about.]

On Sun, 25 Jun 2017 at 01:34 Antoine Pitrou  wrote:

> On Sat, 24 Jun 2017 21:37:46 -0700
> Larry Hastings  wrote:
> > On 06/24/2017 09:14 PM, Serhiy Storchaka wrote:
> > > Not only core developers make PRs for CPython. Since all non-trivial
> > > changes need to be mentioned in Misc/NEWS, blurb becomes a required
> > > tool for all committers.
> >
> > Well, no.  *Writing blurb-compatible files* becomes a required step for
> > all committers.  And blurb makes that easy.  But it's pretty easy to
> > write them by hand; that's why we pre-created the "next" directories,
> > and there are instructions in the dev guide.
>
> Hmm.  If it were so easy, you wouldn't have felt the need to add that
> functionality to blurb, right? :-)
>

Actually a good amount of blurb is to handle blowing up a Misc/NEWS file
into individual news files and then re-combining them back into a Misc/NEWS
file for release. The format also simplified over time while we refined the
design so that the file name contains the most common metadata, thus not
really needing to check the formatting as much.


>
> This is touching a more general problem, though.  Before GitHub, we
> (core developers) would take the patch submitted by a contributor, make
> whatever minor changes were needed (e.g. Misc/NEWS) and push the
> aggregate ourselves.  With GitHub, while it's possible to edit someone
> else's PR, it's frankly a PITA (I've tried to do it once, I don't want
> to try a second time unless GitHub makes it massively easier and less
> footgunning-prone).


So there's editing a PR by adding/removing files and then there's editing
files that have already been touched by the PR. In the latter case you can
just edit the files through a browser while reviewing them no problem (I do
this regularly rather than ask a PR submitter to fix e.g. a spelling
mistake, just click the pencil icon at the top-right corner of the diff for
the file).

In the former case where you want to add/delete a file is where it's less
obvious on how to do it. Donald has said how to pull down a pull request as
a branch in your clone, editing, and then pushing again. But it turns out
you can *also* do it entirely through the web UI. For instance, if you look
at your https://github.com/pitrou/cpython/tree/signal_pyatomic branch you
had for your https://github.com/python/cpython/pull/2417 PR (sorry about
that, but I needed someone's branch to test with that wasn't my own), you
will see I created and deleted a news file, and I did that entirely through
my browser. The biggest pain, though, is navigating back to someone's
branch as I don't think there's a direct link to be found in a PR back to
the repo:branch it's based on.

IOW everything you used to do by hand can still be done by hand, just
through a web browser ... on a tablet at the beach with WiFi (which was one
of my goals of this whole transition to begin with :) .


> So we have to rely on contributors to make the PR
> merge-ready by themselves... which means spending some time guiding
> them through the oh-so-exciting steps necessary to add the right
> Misc/NEWS entry,


My hope is that having a news entry status check through Bedevere with help
with that. I'm also open to ideas on how to make it more obvious to
first-time contributors that things need to be done, e.g. maybe a default
PR message that is nothing more than "" (making it an HTML comment means it won't' ever show
up in the rendered output of the PR, plus that comment is short and thus
easy to ignore)? We could also make the CONTRIBUTING.md file have a
bullet-point list of the key things we expect people to do that is
different from most projects. (The only other thing I can think of is a
comment for first-time contributors pointing all of this out, but I'm leery
of that as all of our bots are stateless and this would make at least
Bedevere stateful.)

And if you want to make it easier to add/remove files in a PR, we could
look at adding a link to the branch the PR comes from so that it's easy to
get to to do things like add a news file through the browser.

If any of this sounds reasonable we can discuss it over on core-workflow.



> or fix the occasional bit of reStructuredText
> mis-syntax.
>
>
You can make those sorts of changes in the browser yourself.

-Brett


> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-26 Thread Brett Cannon 
On Mon, 26 Jun 2017 at 13:38 Victor Stinner 
wrote:

> 2017-06-26 21:58 GMT+02:00 Brett Cannon :
> > I don't see why regrtest isn't the right place for this.
>
> The current regrest CLI isn't designed for subcommands, and I don't
> want to "pollute" regrtest with multiple options for bisect.
> Currently, my script has already 4 options:
>
> haypo@selma$ python3 ~/prog/GIT/misc/python/bisect_test.py --help
> usage: bisect_test.py [-h] [-i INPUT] [-o OUTPUT] [-n MAX_TESTS] [-N
> MAX_ITER]
>
> optional arguments:
>   -h, --helpshow this help message and exit
>   -i INPUT, --input INPUT
> Test names produced by --list-tests written into a
> file. If not set, run --list-tests
>   -o OUTPUT, --output OUTPUT
> Result of the bisection
>   -n MAX_TESTS, --max-tests MAX_TESTS
> Maximum number of tests to stop the bisection
> (default: 1)
>   -N MAX_ITER, --max-iter MAX_ITER
> Maximum number of bisection iterations (default:
> 100)
>
> I really like subcommands, it's a nice way to design complex CLI ;-)
>

Then I think you answered your own question about whether this should go
into regrtest ;) (Unless you want to rewrite regrtest's CLI to support
subcommands.)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-26 Thread Serhiy Storchaka 

26.06.17 23:37, Victor Stinner пише:

2017-06-26 21:58 GMT+02:00 Brett Cannon :

I don't see why regrtest isn't the right place for this.


The current regrest CLI isn't designed for subcommands, and I don't
want to "pollute" regrtest with multiple options for bisect.
Currently, my script has already 4 options:

haypo@selma$ python3 ~/prog/GIT/misc/python/bisect_test.py --help
usage: bisect_test.py [-h] [-i INPUT] [-o OUTPUT] [-n MAX_TESTS] [-N MAX_ITER]

optional arguments:
   -h, --helpshow this help message and exit
   -i INPUT, --input INPUT
 Test names produced by --list-tests written into a
 file. If not set, run --list-tests
   -o OUTPUT, --output OUTPUT
 Result of the bisection
   -n MAX_TESTS, --max-tests MAX_TESTS
 Maximum number of tests to stop the bisection
 (default: 1)
   -N MAX_ITER, --max-iter MAX_ITER
 Maximum number of bisection iterations (default: 100)

I really like subcommands, it's a nice way to design complex CLI ;-)


You could make it just a submodule in the test package.

./python -m test.bisect -R 3:3 test_os

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com