My personal number one measure of module quality is documentation, hands down. Nothing else comes close.
That means a README that provides a good overview of the module’s structure, scope, requirements, and capabilities, and a *thorough* REFERENCE(.md) document. As a module quality consideration, I don’t really care whether the reference documentation is automatically generated. I do care that it covers every public member of the module, accurately and precisely explaining the significance, allowed values, default values, and relationships among them. Strings-derived documentation can be pretty poor, so I don’t attribute much weight to whether docs were generated that way. If I have to look at manifests or other technical artifacts to determine whether the module supports my needs or how to prod it into doing so then that’s a fail in my book. Good documentary comments are high on my list for code quality, too. Consistent code style comes in here as well, but not so much most specific style choices. A good test suite (that passes) also makes the cut, and in it, I would prefer to see both unit tests and functional tests. I don’t particularly object to the voxpupuli puppet-lint checks, but I consider those a weak proxy for a subjective analysis of the module code. I appreciate that most of that is difficult for a computer to evaluate. John Bollinger From: voxpup...@groups.io <voxpup...@groups.io> On Behalf Of David Hollinger via groups.io Sent: Friday, October 29, 2021 9:44 AM To: voxpup...@groups.io Cc: puppet-users@googlegroups.com Subject: Re: [voxpupuli] Do you have opinions on what module quality means? Caution: External Sender. Do not open unless you know the content is safe. I think we are looking at a couple different discussions here: * Module Quality * Code Quality I think the former is always going to be difficult to track in any meaningful way, largely because expectations for what a module should or shouldn't do, or even the how it does something, is going to vary from person to person; org to org. On Module quality, I would say that the Forge should just take a page from the playbook of things like RubyGems, NPMJS, Pypi, etc and just track some combination of: * Homepage * Downloads * Likes/Stars/Forks on GitHub/Gitlab/BitBucket/etc * Open Issues * List of Files that are installed * List of manual downloads * Release history * Additional useful links On Code Quality, I think the following checks should be done: * Arrow Alignment * Indentation * Trailing comma (either have or don't have it - standardization is good) * All the checks included in https://github.com/voxpupuli/voxpupuli-puppet-lint-plugins/blob/master/voxpupuli-puppet-lint-plugins.gemspec<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvoxpupuli%2Fvoxpupuli-puppet-lint-plugins%2Fblob%2Fmaster%2Fvoxpupuli-puppet-lint-plugins.gemspec&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851057833%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=I%2FSmlN1tMXANeU7XowSyYW%2F9Uq5u11Ej7yoD58NwMtA%3D&reserved=0> For tests use badges for the following: * Unit tests with rspec-puppet * Acceptance/Integration tests with Beaker/Litmus/test-kitchen * A README * A generated REFERENCE.md from puppet-strings * Changelog that can be embedded into the Forge page Things that I find too subjective to use a markers for quality: * Use only vendor/OS provided packages: While I can see why some would want this, I personally have not worked at a place that could utilize OS provided packages for a lot of non-OS level software. Things like Nginx, Ruby, Go, Apache, PHP, etc tend to be too old even in Ubuntu to use for running applications and in most cases the OS packages/repos have turned into just what is needed to support running applications that contain or install their own requirements - either via containerization or vendor/project provided repositories. * Don't include binaries: This one is, for me, a hard no. With Modules that include Bolt tasks, you limit what kinds of tasks can be included or what languages those tasks can be written in which could be limiting as there are a lot in the Puppet community not interested in learning Ruby, but want their tasks to do work that can't necessarily be done by bash or powershell scripts. So allowing those users to built tasks in Go, Rust, etc is a no brainer, IMO. Thoughts? --- David Hollinger III Software Engineer WPEngine Sent with ProtonMail<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fprotonmail.com%2F&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851057833%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=KWlq3y32NYPtUbV3ki7Xnva7TeIzylTqUMu%2BfCpsXE4%3D&reserved=0> Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, October 28th, 2021 at 3:33 PM, Ben Ford <ben.f...@puppet.com<mailto:ben.f...@puppet.com>> wrote: Trick question, I know. Having strong opinions is part of why we do the work we do. So I'm hoping that many of you will help us out on this project. Some time ago, the Forge turned off the quality scores detail display because many of the warnings were confusing and frightening to less experienced Puppet users. And we didn't really have a good specification of what made quality code anyway. That's what I hope to improve on today, and we'd like to turn it back on with a better picture of each module's quality. Vox Pupuli has a meta-gem that defines a list of the puppet-lint checks<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvoxpupuli%2Fvoxpupuli-puppet-lint-plugins%2Fblob%2Fmaster%2Fvoxpupuli-puppet-lint-plugins.gemspec&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851067794%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Y%2FlqbI0AptAG66MGn1MITMXJMJzISp7aYiTyBoKohXA%3D&reserved=0> that they use for their definition of a quality Puppet module. I'd like the Forge and Vox Pupuli to be in alignment here, and this seems like the best place to start. Do you have a favorite list of puppet-lint checks you use? Do you think they'd be useful as an ecosystem standard? Can you support that opinion with a cogent argument for it? Suggest it as a pull request and let's have that conversation. Together as a community we can all come up with a solid starting point as a standard definition of module quality. https://github.com/voxpupuli/voxpupuli-puppet-lint-plugins/blob/master/voxpupuli-puppet-lint-plugins.gemspec<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvoxpupuli%2Fvoxpupuli-puppet-lint-plugins%2Fblob%2Fmaster%2Fvoxpupuli-puppet-lint-plugins.gemspec&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851067794%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Y%2FlqbI0AptAG66MGn1MITMXJMJzISp7aYiTyBoKohXA%3D&reserved=0> Thank you all for your help! -- Ben Ford @binford2k Ecosystem Product Manager _._,_._,_ ________________________________ Groups.io Links: You receive all messages sent to this group. View/Reply Online (#398)<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.io%2Fg%2Fvoxpupuli%2Fmessage%2F398&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851077748%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BJrXq4gKiTO1Byic8PXp3fIN3793BBnqAXUuMfu8eIE%3D&reserved=0> | Reply To Group<mailto:voxpup...@groups.io?subject=Re:%20Re%3A%20%5Bvoxpupuli%5D%20Do%20you%20have%20opinions%20on%20what%20module%20quality%20means%3F> | Reply To Sender<mailto:david.hollin...@moduletux.com?subject=Private:%20Re:%20Re%3A%20%5Bvoxpupuli%5D%20Do%20you%20have%20opinions%20on%20what%20module%20quality%20means%3F> | Mute This Topic<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.io%2Fmt%2F86662086%2F861318&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851077748%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=tk4ERvobnT8lOlvRjVT7Oh91Ypt7EXz5aUVo2wDp0Ss%3D&reserved=0> | New Topic<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.io%2Fg%2Fvoxpupuli%2Fpost&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851087707%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4T51dbr9J8SZrwBp4jiKgKAU6qWR3Tq%2Feal%2B6%2BtKok4%3D&reserved=0> Your Subscription<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.io%2Fg%2Fvoxpupuli%2Feditsub%2F861318&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851087707%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=HeVnsqRDfRN5pq%2FN79G8Jflo2uNsqhLv7723ZL9f738%3D&reserved=0> | Contact Group Owner<mailto:voxpupuli+ow...@groups.io> | Unsubscribe<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.io%2Fg%2Fvoxpupuli%2Fleave%2F1933025%2F861318%2F909231714%2Fxyzzy&data=04%7C01%7CJohn.Bollinger%40StJude.org%7Ccf737d31ff9549b6860a08d99aea80b7%7C22340fa892264871b677d3b3e377af72%7C0%7C0%7C637711154851087707%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=C0lzbzF4kUYpvkU67lSJCjdmaXgkSIF4NPu4Lbn1nD0%3D&reserved=0> [john.bollin...@stjude.org] _._,_._,_ ________________________________ Email Disclaimer: www.stjude.org/emaildisclaimer Consultation Disclaimer: www.stjude.org/consultationdisclaimer -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CH2PR04MB695027CDB12E31373A6F2680E0879%40CH2PR04MB6950.namprd04.prod.outlook.com.
