On July 30, 2024 2:47:08 AM UTC, "Louis-Philippe Véronneau" <po...@debian.org>
wrote:
>On 2024-07-30 11:09, Scott Kitterman wrote:
>>
>>
>> On July 30, 2024 12:49:50 AM UTC, "Louis-Philippe Véronneau"
>> <po...@debian.org> wrote:
>>> On 2024-07-29 21:07, Scott Kitterman wrote:
>>>>
>>>>
>>>> On July 29, 2024 8:53:11 AM UTC, "Louis-Philippe Véronneau"
>>>> <po...@debian.org> wrote:
>>>>> Hello,
>>>>>
>>>>> As discussed during the DebConf24 Python BoF, I'm submitting this change
>>>>> to the policy to require the use of the upstream test suite, both during
>>>>> the build process and as an autopkgtest.
>>>>>
>>>>> You can find the MR here:
>>>>>
>>>>> https://salsa.debian.org/python-team/tools/python-modules/-/merge_requests/24
>>>>>
>>>>> People present at the BoF seemed to think this was a good idea. If you
>>>>> don't please reply to this message and make yourself heard :)
>>>>
>>>> I understand the theory and why it's a good idea to run the test suite. I
>>>> don't think it ought to be a hard requirement. I have several packages
>>>> where there's a test suite, but I don't run it:
>>>>
>>>> 1. The largest set is packages that need test only dependencies which are
>>>> not packaged. When I am packaging something new which has a test suite,
>>>> then I generally package any needed test depends. If those test depends
>>>> also need test depends packaged, I generally stop and don't enable tests
>>>> for things that are only in the archives to support tests. Noseofyeti is
>>>> an example of this.
>>>
>>> That sounds like a valid technical reason not to run the tests to me :)
>>>
>>>> 2. There's at least one case where Debian has customizations that cause
>>>> the test suite to fail, but the failures don't seem to cause any real
>>>> problems. If anyone wants to make it so the weasyprint test suite works
>>>> on Debian, please knock yourself out.
>>>
>>> Again, as long as you document that, I don't think it would go against the
>>> proposed policy change.
>>>
>>>> 3. I also maintain multiple packages which require network access to run
>>>> their test suite, so they can't run tests during build, only autopkgtests.
>>>
>>> Same.
>>>
>>
>> Except for #3, I don't get that from the wording in the MR. I don't think
>> not worth the trouble is a technical reason. I think the real rule that's
>> being proposed is that packages must run the test suit or document why not.
>> I don't have a problem with that, but I don't think that's what it actually
>> says now. I think if you were to change must to should and strike the word
>> technical before reason, it would accomplish the same thing and be clearer.
>> I could support that.
>
>Language is hard and I'm not a native English speaker :)
>
>What I want to prevent is people not running tests because they don't feel
>like it, even though it would not take them a large amount of efforts to do so.
>
>I'll strike "technical", as it seems others also interpreted this word they
>way you have.
>
>As for "MUST" vs "SHOULD", I believe the exception clause provides enough
>leeway to justify a reasonable way out in case of $VALID_REASON.
>
>"SHOULD" is not particularly strong and again, I fear it would let people get
>away with not running tests although it wouldn't be much work to do so...
>
I guess it depends on how you look at the word. In the context of technical
specifications, I tend to think of terms as defined in RFC 2119 [1]. I think
that's pretty close to what you are suggesting:
3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.
The way RFC 2119 defines MUST doesn't leave much room for exceptions. If you
want to be clear, I suggest SHOULD with a reference to RFC 2119. While not
universal, it is a widely used reference for definitions of these terms in
technical specifications.
Scott K
[1] https://datatracker.ietf.org/doc/html/rfc2119#section-3
