Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-19 Thread Vít Ondruch
Dne 17.10.2016 v 11:16 Pavol Babincak napsal(a): > Whereas tests repositories have only master branch. I spoke to Pavol today a bit and I realized that I missed this peace of the information, which comes quite essential to me and worth of highlighting. This is very important, since if there i

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-18 Thread Vít Ondruch
Dne 17.10.2016 v 17:42 Tim Flink napsal(a): > On Mon, 17 Oct 2016 11:16:22 +0200 > Pavol Babincak wrote: > >> On 10/17/2016 06:46 AM, Tim Flink wrote: >>> On Mon, 3 Oct 2016 13:50:33 -0600 >>> Tim Flink wrote: >>> One of the features for Taskotron that we've been planning since the

Re: RFC: Storing Automated Tasks/Tests In Dist-Git (git-submodules)

2016-10-18 Thread Pavel Raiskup
On Tuesday, October 18, 2016 10:01:30 AM CEST Pierre-Yves Chibon wrote: > > I'm not against it, as I'm not going to hack that :) but this is a lot of > > expensive complexity, when submodules are here clearly for this purpose. > > And are you going to hack on submodules? (both as user and to help

Re: RFC: Storing Automated Tasks/Tests In Dist-Git (git-submodules)

2016-10-18 Thread Pierre-Yves Chibon
On Tue, Oct 18, 2016 at 09:37:45AM +0200, Pavel Raiskup wrote: > On Wednesday, October 5, 2016 11:40:44 AM CEST Tim Flink wrote: > > I didn't notice that my reply went only to Pavel, resending to devel@ > > > > On Tue, 04 Oct 2016 10:25:46 +0200 > > Pavel Raiskup wrote: > > > > > On Monday, Octo

Re: RFC: Storing Automated Tasks/Tests In Dist-Git (git-submodules)

2016-10-18 Thread Pavel Raiskup
On Wednesday, October 5, 2016 11:40:44 AM CEST Tim Flink wrote: > I didn't notice that my reply went only to Pavel, resending to devel@ > > On Tue, 04 Oct 2016 10:25:46 +0200 > Pavel Raiskup wrote: > > > On Monday, October 3, 2016 1:50:33 PM CEST Tim Flink wrote: > > > https://phab.qadevel.cloud

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Miroslav Suchý
Dne 17.10.2016 v 06:46 Tim Flink napsal(a): > Which brings me to the question that I'd like to get some feedback > on: would it be preferable to store checks/tests within directories of > existing dist-git repos or create a new namespace to store checks/tests > and fiddle around with tooling etc.

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Matthew Miller
On Mon, Oct 17, 2016 at 11:57:03AM -0600, Tim Flink wrote: > If we do keep going toward the goal of having more automation support > for testing and gating builds from koji based on results from that > automation, I don't understand how it makes sense to let more people > have write access to the c

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Matthew Miller
On Mon, Oct 17, 2016 at 05:18:25PM +, Zbigniew Jędrzejewski-Szmek wrote: > > > those tests to pass to be submitted and merged as a single pull request. > > > I'd love to see a PR that adds a test for one of my packages, exposes > > > some bugs, but immediately fixes any fallout. I would be less

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Tim Flink
On Mon, 17 Oct 2016 11:56:35 -0400 Matthew Miller wrote: > On Mon, Oct 17, 2016 at 09:42:51AM -0600, Tim Flink wrote: > > One of the differences in Fedora is that I expect most check/test > > contributions will come from package maintainers instead of > > dedicated QA folks. At this time, there j

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Tim Flink
On Mon, 17 Oct 2016 17:18:25 + Zbigniew Jędrzejewski-Szmek wrote: > On Mon, Oct 17, 2016 at 12:45:30PM -0400, Matthew Miller wrote: > > On Mon, Oct 17, 2016 at 04:38:28PM +, Zbigniew > > Jędrzejewski-Szmek wrote: > > > It's a good principle to require both tests and fixes required for >

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Oct 17, 2016 at 12:45:30PM -0400, Matthew Miller wrote: > On Mon, Oct 17, 2016 at 04:38:28PM +, Zbigniew Jędrzejewski-Szmek wrote: > > It's a good principle to require both tests and fixes required for > > those tests to pass to be submitted and merged as a single pull request. > > I'd

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Matthew Miller
On Mon, Oct 17, 2016 at 04:38:28PM +, Zbigniew Jędrzejewski-Szmek wrote: > It's a good principle to require both tests and fixes required for > those tests to pass to be submitted and merged as a single pull request. > I'd love to see a PR that adds a test for one of my packages, exposes > some

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Oct 17, 2016 at 11:56:35AM -0400, Matthew Miller wrote: > On Mon, Oct 17, 2016 at 09:42:51AM -0600, Tim Flink wrote: > > One of the differences in Fedora is that I expect most check/test > > contributions will come from package maintainers instead of dedicated > > QA folks. At this time, th

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Matthew Miller
On Mon, Oct 17, 2016 at 09:42:51AM -0600, Tim Flink wrote: > One of the differences in Fedora is that I expect most check/test > contributions will come from package maintainers instead of dedicated > QA folks. At this time, there just aren't enough available person hours > among the Fedora QA folk

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Tim Flink
On Mon, 17 Oct 2016 11:16:22 +0200 Pavol Babincak wrote: > On 10/17/2016 06:46 AM, Tim Flink wrote: > > On Mon, 3 Oct 2016 13:50:33 -0600 > > Tim Flink wrote: > > > >> One of the features for Taskotron that we've been planning since > >> the beginning was a way for contributors to maintain the

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-17 Thread Pavol Babincak
On 10/17/2016 06:46 AM, Tim Flink wrote: On Mon, 3 Oct 2016 13:50:33 -0600 Tim Flink wrote: One of the features for Taskotron that we've been planning since the beginning was a way for contributors to maintain their own automated tasks/tests which would be run during a package's lifecycle. I'

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-16 Thread Tim Flink
On Mon, 3 Oct 2016 13:50:33 -0600 Tim Flink wrote: > One of the features for Taskotron that we've been planning since the > beginning was a way for contributors to maintain their own automated > tasks/tests which would be run during a package's lifecycle. > > I'm happy to say that we're almost t

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-06 Thread Matthew Miller
On Thu, Oct 06, 2016 at 12:44:02PM +, Zbigniew Jędrzejewski-Szmek wrote: > Maybe we should standardize the prefix for git commits: "taskotron: " > or "tests: " or whatever, so it's easy to stop in git history. That'll have to be automated somehow or I don't think people will remember. -- Ma

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-06 Thread Zbigniew Jędrzejewski-Szmek
On Wed, Oct 05, 2016 at 12:16:32PM -0600, Tim Flink wrote: > On Tue, 4 Oct 2016 09:19:15 -0400 > Matthew Miller wrote: > > > On Mon, Oct 03, 2016 at 02:35:00PM -0600, Kevin Fenzi wrote: > > > Another alternate here is that we could make taskotron a 'namespace' > > > like currently rpms/ and docke

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Tim Flink
On Tue, 4 Oct 2016 09:19:15 -0400 Matthew Miller wrote: > On Mon, Oct 03, 2016 at 02:35:00PM -0600, Kevin Fenzi wrote: > > Another alternate here is that we could make taskotron a 'namespace' > > like currently rpms/ and docker/ are. Then we would have > > perhaps: /taskotron/rpms/foobar/ as the

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Tim Flink
On Tue, 4 Oct 2016 11:08:36 -0600 Kevin Fenzi wrote: > > That being said, we're fine with either storage paradigm and it > > doesn't matter much if we look for tasks in a directory inside > > dist-git branches or a separate repo which only holds tasks as long > > as there is a single convention

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Tim Flink
On Tue, 4 Oct 2016 15:07:28 +0200 Vít Ondruch wrote: > Hi Tim, > > How about this use case: > > Let say I have Ruby on Rails. This framework is much broader then one > rubygem-rails package. Where test for such framework will be stored? Honestly, it depends on where you want those test cases t

Re: RFC: Storing Automated Tasks/Tests In Dist-Git (git-submodules)

2016-10-05 Thread Tim Flink
I didn't notice that my reply went only to Pavel, resending to devel@ On Tue, 04 Oct 2016 10:25:46 +0200 Pavel Raiskup wrote: > On Monday, October 3, 2016 1:50:33 PM CEST Tim Flink wrote: > > https://phab.qadevel.cloud.fedoraproject.org/w/taskotron/new_distgit_task_storage_proposal/ > > ... > >

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Tim Flink
On Tue, 4 Oct 2016 10:25:26 +0200 Jakub Jelen wrote: > On 10/03/2016 09:50 PM, Tim Flink wrote: > > One of the features for Taskotron that we've been planning since the > > beginning was a way for contributors to maintain their own automated > > tasks/tests which would be run during a package's l

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Tim Flink
On Tue, 4 Oct 2016 20:09:14 +0100 "Richard W.M. Jones" wrote: > On Mon, Oct 03, 2016 at 08:21:42PM +, Zbigniew Jędrzejewski-Szmek > wrote: > > On Mon, Oct 03, 2016 at 01:50:33PM -0600, Tim Flink wrote: > > > One of the features for Taskotron that we've been planning since > > > the beginnin

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Petr Pisar
On 2016-10-05, Pavel Raiskup wrote: > Right, or more - it would be fine to have testsuite `make install`able > (automatically by Automake e.g.). Then anybody could create 'foo-test' > package and than end-users could do the testing themselves too, > transparently - the same way as taskotron would

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Pavel Raiskup
On Wednesday, October 5, 2016 10:49:26 AM CEST Dan Horák wrote: > I haven't thought much about it yet how feasible it would be, but why > not "abuse" rpm infrastructure even for the tests? Introduce foo-test > package as a test-suite for package foo and do everything within its > spec file - instal

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Dan Horák
On Wed, 5 Oct 2016 09:32:49 +0100 "Richard W.M. Jones" wrote: > On Wed, Oct 05, 2016 at 07:54:59AM +0200, Pavel Raiskup wrote: > > On Tuesday, October 4, 2016 8:09:14 PM CEST Richard W.M. Jones > > wrote: > > > And related to this question, do we also need to define > > > "TestRequires" packages/

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-05 Thread Richard W.M. Jones
On Wed, Oct 05, 2016 at 07:54:59AM +0200, Pavel Raiskup wrote: > On Tuesday, October 4, 2016 8:09:14 PM CEST Richard W.M. Jones wrote: > > And related to this question, do we also need to define > > "TestRequires" packages/dependencies? > > Sounds like natural approach would be to install the buil

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-04 Thread Pavel Raiskup
On Tuesday, October 4, 2016 8:09:14 PM CEST Richard W.M. Jones wrote: > And related to this question, do we also need to define > "TestRequires" packages/dependencies? Sounds like natural approach would be to install the built packages into some minimal environment, and the packages itself should

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-04 Thread Richard W.M. Jones
On Mon, Oct 03, 2016 at 08:21:42PM +, Zbigniew Jędrzejewski-Szmek wrote: > On Mon, Oct 03, 2016 at 01:50:33PM -0600, Tim Flink wrote: > > One of the features for Taskotron that we've been planning since the > > beginning was a way for contributors to maintain their own automated > > tasks/tests

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-04 Thread Kevin Fenzi
On Mon, 3 Oct 2016 15:57:14 -0600 Tim Flink wrote: > Thanks for the clarification - the emphasis was on coming support for > PRs. I've reworded that part of the proposal to make it more clear > that dist-git isn't "moving to pagure". Thanks. > > > Another alternate here is that we could make t

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-04 Thread Matthew Miller
On Mon, Oct 03, 2016 at 02:35:00PM -0600, Kevin Fenzi wrote: > Another alternate here is that we could make taskotron a 'namespace' > like currently rpms/ and docker/ are. Then we would have > perhaps: /taskotron/rpms/foobar/ as the top level and all the rest is > the same. This would get us a sepe

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-04 Thread Vít Ondruch
Hi Tim, How about this use case: Let say I have Ruby on Rails. This framework is much broader then one rubygem-rails package. Where test for such framework will be stored? How about tests, which might cover multiple versions of components? Lets say I will have some generic test cases which shoul

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-04 Thread Jakub Jelen
On 10/03/2016 09:50 PM, Tim Flink wrote: One of the features for Taskotron that we've been planning since the beginning was a way for contributors to maintain their own automated tasks/tests which would be run during a package's lifecycle. I'm happy to say that we're almost to this milestone and

Re: RFC: Storing Automated Tasks/Tests In Dist-Git (git-submodules)

2016-10-04 Thread Pavel Raiskup
On Monday, October 3, 2016 1:50:33 PM CEST Tim Flink wrote: > https://phab.qadevel.cloud.fedoraproject.org/w/taskotron/new_distgit_task_storage_proposal/ > ... > Please read through the wiki page I mentioned above and give us > feedback on whether what we're planning to implement is going to be > u

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-03 Thread Tim Flink
On Mon, 3 Oct 2016 14:35:00 -0600 Kevin Fenzi wrote: > On Mon, 3 Oct 2016 13:50:33 -0600 > Tim Flink wrote: > > > One of the features for Taskotron that we've been planning since the > > beginning was a way for contributors to maintain their own automated > > tasks/tests which would be run duri

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-03 Thread Tim Flink
On Mon, 3 Oct 2016 20:21:42 + Zbigniew Jędrzejewski-Szmek wrote: > On Mon, Oct 03, 2016 at 01:50:33PM -0600, Tim Flink wrote: > > One of the features for Taskotron that we've been planning since the > > beginning was a way for contributors to maintain their own automated > > tasks/tests which

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-03 Thread Tim Flink
On Mon, 3 Oct 2016 21:11:58 +0100 "Richard W.M. Jones" wrote: > On Mon, Oct 03, 2016 at 01:50:33PM -0600, Tim Flink wrote: > > One of the features for Taskotron that we've been planning since the > > beginning was a way for contributors to maintain their own automated > > tasks/tests which would

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-03 Thread Kevin Fenzi
On Mon, 3 Oct 2016 13:50:33 -0600 Tim Flink wrote: > One of the features for Taskotron that we've been planning since the > beginning was a way for contributors to maintain their own automated > tasks/tests which would be run during a package's lifecycle. > > I'm happy to say that we're almost t

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-03 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Oct 03, 2016 at 01:50:33PM -0600, Tim Flink wrote: > One of the features for Taskotron that we've been planning since the > beginning was a way for contributors to maintain their own automated > tasks/tests which would be run during a package's lifecycle. > > I'm happy to say that we're al

Re: RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-03 Thread Richard W.M. Jones
On Mon, Oct 03, 2016 at 01:50:33PM -0600, Tim Flink wrote: > One of the features for Taskotron that we've been planning since the > beginning was a way for contributors to maintain their own automated > tasks/tests which would be run during a package's lifecycle. > > I'm happy to say that we're al

RFC: Storing Automated Tasks/Tests In Dist-Git

2016-10-03 Thread Tim Flink
One of the features for Taskotron that we've been planning since the beginning was a way for contributors to maintain their own automated tasks/tests which would be run during a package's lifecycle. I'm happy to say that we're almost to this milestone and wanted to get some feedback from devel@ on