Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-17 Thread glob
thanks to everyone who provided feedback for this proposal. looks like general consensus is to proceed with the plan. i've filed bug 1454867 to track the work. -glob glob wrote: The plan is to create a YAML file for each library containing metadata such as the homepage url, vendored version,

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Byron Jones
Mark Banner wrote: There are some directories where we only import a file from a third-party and it is alongside other files in that same directory, e.g. testing/modules has "ajv-4.1.1.js" and "sinon-2.3.2.js" imported from elsewhere. How do they fit into this scheme? they would have to be

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Byron Jones
Axel Hecht wrote: One thing I'm missing is the ability to do mono-repo imports. Say we want to vendor in https://github.com/projectfluent/fluent.js/tree/master/fluent-gecko. i'm not sure i understand the specific concerns around mono-repos; you can use exclude/include to perform a selective i

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Thom Chiovoloni
On Tue, Apr 10, 2018 at 6:00 AM, Mark Banner wrote: > There are some directories where we only import a file from a third-party > and it is alongside other files in that same directory, e.g. > testing/modules has "ajv-4.1.1.js" and "sinon-2.3.2.js" imported from > elsewhere. > > How do they fit i

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Martin Thomson
On Tue, Apr 10, 2018 at 6:41 AM, glob wrote: >> You don't permit the use of a tag for vendoring, is that intentional? > > to echo gps and mike's responses use of a sha to is preferred over tags. Maybe. We currently use tags. Think about the usage model. If the process is to author the YAML, th

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Axel Hecht
A couple of comments: One thing I'm missing is the ability to do mono-repo imports. Say we want to vendor in https://github.com/projectfluent/fluent.js/tree/master/fluent-gecko. For js libraries, we might also want to pay attention to .npmignore (others already mentioned hg, so also .hgignor

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Nicholas Alexander
Hi all, On Mon, Apr 9, 2018 at 9:25 PM, glob wrote: > mozilla-central contains code vendored from external sources. Currently > there is no standard way to document and update this code. In order to > facilitate automation around auditing, vendoring, and linting we intend to > require all vendor

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread glob
Tom Tromey wrote: this should be: https://goo.gl/QZyz4x for the full specification. Some code in DevTools is vendored by dropping webpack bundles into the tree. The bundles are created by running a yarn command in the source repository; this also copies the bundle into an M-C tree. If these dir

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Tom Ritter
I've spoken to glob about this offline; but just wanted to note: Our fledgling 'Third Party Library Audit' project is planning on using this metadata (even if the library itself isn't completely vendored) for checking for security issues in upstream and auto-filing bugs. -tom On Mon, Apr 9, 2018

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Tom Tromey
>> this should be: https://goo.gl/QZyz4x for the full specification. Some code in DevTools is vendored by dropping webpack bundles into the tree. The bundles are created by running a yarn command in the source repository; this also copies the bundle into an M-C tree. If these directories are goin

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread James Graham
On 10/04/2018 14:34, Ted Mielczarek wrote: On Tue, Apr 10, 2018, at 9:23 AM, James Graham wrote: On 10/04/2018 05:25, glob wrote: mozilla-central contains code vendored from external sources. Currently there is no standard way to document and update this code. In order to facilitate automation

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Ted Mielczarek
On Tue, Apr 10, 2018, at 9:23 AM, James Graham wrote: > On 10/04/2018 05:25, glob wrote: > > mozilla-central contains code vendored from external sources. Currently > > there is no standard way to document and update this code. In order to > > facilitate automation around auditing, vendoring, and

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread glob
James Graham wrote: So we now have moz.build that in addition to build instructions, contains metadata for mozilla-authored code (e.g. bugzilla components) and moz.yaml that will contain similar metadata but only for non-mozilla-authored code, as well as Cargo.toml that will contain (some of)

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread James Graham
On 10/04/2018 05:25, glob wrote: mozilla-central contains code vendored from external sources. Currently there is no standard way to document and update this code. In order to facilitate automation around auditing, vendoring, and linting we intend to require all vendored code to be annotated wi

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread Mark Banner
On 10/04/2018 05:25, glob wrote: The plan is to create a YAML file for each library containing metadata such as the homepage url, vendored version, bugzilla component, etc. See https://goo.gl/QZyz4xfor the full specification. There are some directories where we only import a file from a third-p

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-10 Thread glob
Henri Sivonen wrote: This proposal makes sense to me when it comes to libraries that are not vendored from crates.io. However, this seems very heavyweight and only adds the Bugzilla metadata for crates.io crates. It seems to me that declaring the Bugzilla component isn't worth the trouble of havi

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread Henri Sivonen
On Tue, Apr 10, 2018 at 7:33 AM, Byron Jones wrote: > glob wrote: >> >> The plan is to create a YAML file for each library containing metadata >> such as the homepage url, vendored version, bugzilla component, etc. See >> https://goo.gl/QZyz4xfor the full specification. > > > this should be: https

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread Byron Jones
this should be: https://goo.gl/QZyz4x for the full specification. This format is essentially assuming the vendored code comes from a VCS repository. We have plenty of third party code that is imported through upstream tarballs, so this should probably be accounted for. we can certainly support

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread Byron Jones
this should be: https://goo.gl/QZyz4x for the full specification. This format is essentially assuming the vendored code comes from a VCS repository. We have plenty of third party code that is imported through upstream tarballs, so this should probably be accounted for. we can certainly support

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread glob
thanks for the feedback martin, Please consider adding hg.mozilla.org to your list of things you will clone from. adding support to vendor from hg.m.o is a great suggestion, and shouldn't be problematic once the work has been proven with github. You don't permit the use of a tag for vendoring,

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread Gregory Szorc
On Mon, Apr 9, 2018 at 9:52 PM, Mike Hommey wrote: > On Tue, Apr 10, 2018 at 02:46:40PM +1000, Martin Thomson wrote: > > This seems like a good idea. > > > > Please consider adding hg.mozilla.org to your list of things you will > > clone from. That will allow us to remove some ugly hacks from th

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread Mike Hommey
On Tue, Apr 10, 2018 at 02:46:40PM +1000, Martin Thomson wrote: > This seems like a good idea. > > Please consider adding hg.mozilla.org to your list of things you will > clone from. That will allow us to remove some ugly hacks from the > tree for vendoring NSS and NSPR. (libffi uses the same sc

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread Mike Hommey
On Tue, Apr 10, 2018 at 12:33:27PM +0800, Byron Jones wrote: > glob wrote: > > The plan is to create a YAML file for each library containing metadata > > such as the homepage url, vendored version, bugzilla component, etc. See > > https://goo.gl/QZyz4xfor the full specification. > > this should be

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread Martin Thomson
This seems like a good idea. Please consider adding hg.mozilla.org to your list of things you will clone from. That will allow us to remove some ugly hacks from the tree for vendoring NSS and NSPR. (libffi uses the same script, but it seems to be on GitHub now, so that seems like an easy win ass

Re: Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread Byron Jones
glob wrote: The plan is to create a YAML file for each library containing metadata such as the homepage url, vendored version, bugzilla component, etc. See https://goo.gl/QZyz4xfor the full specification. this should be: https://goo.gl/QZyz4x for the full specification. -- glob — engineering

Intent To Require Manifests For Vendored Code In mozilla-central

2018-04-09 Thread glob
mozilla-central contains code vendored from external sources. Currently there is no standard way to document and update this code. In order to facilitate automation around auditing, vendoring, and linting we intend to require all vendored code to be annotated with an in-tree YAML file, and for