Follow-up Comment #19, bug #67372 (group groff): [comment #17 comment #17:] > You are saying there is no regression because only undefined > behaviour changed,
Yes. > and the mandoc(1) test suite is over-testing. I wouldn't phrase it that broadly. I think _mandoc_'s test suite is testing the specifics _of_ undefined behavior. As I said in comment #15, "you could not be so particular about how the formatter recovers from syntactically invalid input. You might run those cases through `groff -Wall -wdelim` and look for non-empty output to the standard error stream." I don't think it is at all a bad thing for _mandoc_ to check up on _groff_ insofar as it sees if _groff_ is correctly detecting and diagnosing invalid input (when told to do so--I don't agree with GNU _troff_'s blinkered default warning mask). But I believe it's fairly well understood that once one encounters undefined behavior in a Turing-complete language, it is futile to write prescriptions for recovery to the interpreter. > So, the first question is whether \w\*v is indeed undefined. > I believe the main place where to look for an answer is: > https://www.gnu.org/software/groff/manual/groff.html.node/Using-Escape-Sequenc > es.html > > Citing selectively, Indeed, as I think the more probative place is "[https://www.gnu.org/software/groff/manual/groff.html.node/Delimiters.html Delimiters]". > > Whereas requests must occur on control lines, escape sequences > can occur intermixed with text and may appear in arguments > to requests, macros, and other escape sequences. > [...] Escape sequence interpolation is of higher precedence > than escape sequence argument interpretation. This rule affords > flexibility in using escape sequences to construct parameters > to other escape sequences. > > Admittedly, this is not completely conclusive, so the judge must > interpret it resorting to legislative intent, or precedent, > or analogy, or common sense. Legislative intent seems to > contrast requests, which can only be invoked in one very particular > place, to escape sequences, which can be invoked (almost?) anywhere. > Formally, it may be possible to interpret a long list that, on > first sight, seems all-encompassing, in particular when it does > not even allude to a single restriction, as "this is intended > as an exhaustive enumeration, nothing else is allowed" - but that > doesn't seem particularly compelling. A canon of interpretation frequently applied in engineering is that "specific claims override general ones". Some of the foregoing language you've quoted is my work, so I must shoulder some of the responsibility for our conflicting interpretations. Not most, however, as you appear not to have read that language prior to composing the "A1.in" test input. In fact, I might not have _written_ the quoted language yet at that point. I did already acknowledge in "task 1" of comment #5 that our manual could be improved in this respect. > Precedent is unambigous from the dawn of time to groff-1.22.4. On the other hand, this is an appeal to Hyrum's Law and therefore not very convincing to me. If we accept Hyrum's Law, there is no point undertaking further development on any software project once it is released. > Several examples also support the idea that the first sentence is > not intended as exhaustive. > > For example: > > * .ds s B > .\*s bold text > invokes the B macro, printing bold text The formatter is not expecting a delimiter where this escape sequence occurs. > * .\*sR bold text > invokes the BR macro, printing only bold in bold The formatter is not expecting a delimiter where this escape sequence occurs. > * .R\*s text bold > invokes the RB macro, printing only bold in bold The formatter is not expecting a delimiter where this escape sequence occurs. > * .ds t bold text > .RB \*t > passes *two* arguments, printing only bold in bold The formatter is not expecting a delimiter where this escape sequence occurs. > * .ds t ld te > .BR bo\*txt > passes *two* arguments, printing only bold in bold The formatter is not expecting a delimiter where this escape sequence occurs. > Now arguably, the wording "may appear in arguments" > might include escape sequences contributing to multiple > arguments - though the more natural interpretation of > that wording would be "may appear in all arguments" > (i.e. any argument may invoke escape sequences.) I think that is beside the point. You may have noticed from the foregoing that the state in the big GNU _troff_ parser's finite state machine that we can label "delimiter character pending" has special rules associated with it. > Even if you argue that the first sentence *is* exhaustive > and you are free to break all the above examples (and more), > the question remains open whether "argument interpretation" > includes argument delimiter interpretation - i would maybe > argue that it does. It can't--not with retention of the "input level" feature. > At least, that is supported by examples > like the following: > > * .ds s [B] > \f\*stext > > * .ds s (CB > \f\*stext > Both print the text in bold. The formatter is not expecting a delimiter where the string interpolation escape sequence occurs. Only some escape sequences accept delimiters. `\f` is not one of them. It's probably far too late and far too deep a change to make all escape sequences in GNU _troff_ delimited, but if we did, its input language would look a lot more like TeX. > This section 5.6.4 mentions none of the terms "interpolation depth", > "nesting depth", or "input level", nor could i find any definition > of these terms anywhere (by grepping over all of groff git master). $ git grep -A2 "nesting depth" doc man doc/groff.texi.in:problem because it tracks the nesting depth of interpolations. doc/groff.texi.in-@xref{Implementation Differences}. doc/groff.texi.in- -- doc/groff.texi.in:@code{troff} keeps track of the nesting depth of escape sequence doc/groff.texi.in-interpolations, so the only characters you need to avoid using as doc/groff.texi.in-delimiters are those that appear in the arguments you input, not any -- man/groff.7.man:keeps track of the nesting depth of escape sequence interpolations, man/groff.7.man-so the only characters you need to avoid using as delimiters are those man/groff.7.man-that appear in the arguments you input, -- man/groff_diff.7.man:or that will be interpolated at varying macro nesting depths. man/groff_diff.7.man-. man/groff_diff.7.man-. -- man/groff_diff.7.man:keeps track of the nesting depth of escape sequence man/groff_diff.7.man-interpolations and other uses of delimiters, man/groff_diff.7.man-as in the I've got a pending commit in my working copy (not reflected in the foregoing "grep"). commit 626ed5b2f339e3ea8a93978f0f4cdecd81918f5c Author: G. Branden Robinson <g.branden.robin...@gmail.com> Date: Tue Jul 29 02:12:26 2025 -0500 doc/groff.texi.in: Tweak "nesting depth" stuff. * "Nesting" manifests in several contexts. Consistently say "nesting depth of XXX". * It's not simply "arguments" (to delimited escape sequences) that possess a nesting depth; the concept is more general to GNU troff's grammar. (I reckon it's common to macro language processors that admit recursion.) * Improve concept index; add entries relating to "input level" (the source code's term regarding escape sequence interpolation) and "interpolation depth" (a term I've thrown around casually in Savannah tickets and other discussions). * Break input lines consistently with our man(7) documents. * Tighten wording. Prompted by discussion with Ingo Schwarze in Savannah #67372. > Unless i'm missing something, they are only mentioned in passing > in obscure corners of the documentation (mostly about compatibility > with historic implementations) without ever properly defining them > and without ever specifying what their effect is, which leads me > to consider them implementation details that the user should not > have to worry about. "Nesting depth of interpolations" appears to be the preponderant wording, and I might need to reform my personal parlance in Savannah tickets and other discussions to align with that. And your final observation is a good one. We are at some pains in _groff_'s manual to avoid discoursing on implementation details of the formatter where we can avoid it. For example, we avoid talking about the "input stack" (which is implicated here) except when discussing the "backtrace" request that iterates through it, or the "slimit" register that can impose a limit on its size. In those cases, there's no escape--we must mention the thing. > I admit that the original idea behind "interpolation depth" > looks interesting, i.e. getting \w'\*s' to work for arbitrary s, > including s containing single quotes, but it appears it was never > really thought through nor designed consistently, let alone > implemented consistently. That's stronger than I'd put it, but I do think it accounts for the argument in this ticket. > It *may* be possible to define and implement it in a consistent > manner as follows: > > 1. If an escape sequence function character that (optionally or > mandatorily) takes a delimited argument is immediately followed > by a valid delimiter that does not result from interpolation, > than any instances of the delimiter resulting from interpolation > while parsing the argument do not end the argument. > 2. If an escape sequence function character that (optionally or > mandatorily) takes a delimited argument is immediately > followed by the beginning of another escape sequence, > rule 1 does not apply. I'll have to ponder on this--it requires more thought than even a second- or third-tier ticket discussion normally commands. > That would possibly provide the intended benefit for \w'\*v' > without breaking \w\*v. I question the value of retaining the semantics of `\w\*v` as observed by _mandoc_'s test suite. Incidentally, I misspoke slightly earlier when enumerating escape sequences implicated by the commit located in my bisection. `\B` is not implicated...but on the other hand `\Z` is. You may also be interested to note this related NEWS item. $ git grep -A2 'no longer accepts a newline' NEWS NEWS:* GNU troff no longer accepts a newline as a delimiter for the NEWS- parameterized escape sequences `\A`, `\b`, `\o`, `\w`, `\X`, and NEWS- `\Z`. That's a pending 1.24.0 change. It's been in the tree for...[checking]...ah, almost a year. Here's the commit. 2024-08-19 G. Branden Robinson <g.branden.robin...@gmail.com> [troff]: Fix Savannah #63142 (newlines as delims). Withdraw support for newlines as escape sequence delimiters. Only a few (6) supported this, and the rationale behind it is unknown. DWB 3.3 troff didn't behave this way in any case. * src/roff/troff/input.cpp (do_overstrike, do_bracket) (do_name_test, do_zero_width_output, do_width) (do_device_control): Do it. As a bonus, check starting delimters for these escape sequences (`\[obAZwX]`) for validity in general. (do_overstrike, do_bracket, do_zero_width_output): Avoid leaking memory when returning early; delete the `new` node we just allocated. (token:next): Be prepared for the `\b` and `\o` escape sequences to return a null pointer (if the sequence doesn't even validly get off the ground), as handlers for `\Z` and `\X` already are. * src/roff/groff/tests/\ some_escapes_accept_newline_delimiters.sh: Delete. * src/roff/groff/groff.am (groff_TESTS): Drop test. * doc/groff.texi.in (Delimiters): * man/groff.7.man (Delimiters): * NEWS: De-document support for newlines as delimiters. Fixes <https://savannah.gnu.org/bugs/?63142>. [snipping opinions on _sh_ and _roff_ as languages] > For now, i'm *not* planning to commit my revert patch to groff, > mostly because a revert is adequate when there is consensus that > there is a regression, but there is no such consensus. Okay. > Besides, i don't currently have a working port of groff from git, > and i remember from the last time that i tried to build one, it > was extremely difficult to get it to build at all, Yuck. I'm not happy to hear that. I'll be interested to hear about anything I can do to coax the _gnulib_ developers into better OpenBSD support. :) > and i neither > want to get distracted right now from working on 1.23 nor commit > anything without proper testing. Repeatedly getting distracted > with git-master during the last two years was one contributing > factor to the intolerable delay dealing with 1.23, so not > getting near git-master is part of my (at this point, close to > desparate) strategy for finally getting 1.23 out of the door. That sounds reasonable to me. > So i'll temporarily use my revert patch in the OpenBSD port - > not because i'm convinced that is the proper long-term solution, > but as an interim stopgap until we develop a consensus what > the intended behaviour actually is. Once groff grows sane and > consistent behaviour, i'm not opposed to changing mandoc(1) > yet again. I plan to commit a further development of my patch in comment #11, meaning applying it to all delimited escape sequences. If it's not too much work and doesn't break _groff_'s own test suite, that is. If it survives that, I'd like for it to go into 1.24.0.rc1 at least so we can see if it causes real-world problems. If real-world problems arise, then rather than reverting it, I'd be more inclined to change it to some kind of "warning: you're using UB" diagnostic, but then proceeding with the behavior as of today. I'm not enthusiastic about your patch because my change really was to make delimiter handling more consistent among supported escape sequences. You might have been less cross with me if you'd also tested `\b`, `\X`, and `\Z` as thoroughly as you did `\A`, because I'll bet you would have encountered mutual inconsistencies among them. > One remark regarding over-testing. You are right that over-testing > can be a real problem. For example, that is the reason why the > mandoc(1) testsuite does not do any -T ps tests just yet: > i simply couldn't figure out any way so far how to avoid that > all PostScript tests would contain more over-testing than > intentional testing. > > But testing only documented features is (sadly) not an option > for groff-mandoc compatibility testing, for multiple reasons. > > 1. Even though groff documentation is significantly better than > lots of other software documentation (that was already true > during wl@ times), and has continued to improve over time, in > particular during the last couple of years, it does still have > its gaps and ambiguities, as this very ticket demonstrates. > When documentation is less than perfect, testing for some > undocumented features is unavoidable. Yes. But it's not always to correct to decide that a test's expectations are warranted. > 2. Manual page authors are generally not known for diligently > reading groff documentation before writing or maintaining > manual pages. My experience has led me to question how much time man page authors in general spend reading any documentation of any sort, on any subject. > Even the authors of the vast majority of > man(7) code generators are not known for that - probably > with the exception of the author(s) of pod2man(1), which > quite reliably produces decent code. Agreed. Quite a cult of ignorance has built up. > But even a generally exceptionally diligent and prolific manual > page author as jmc@ has told me "if what i write doesn't work > as i hoped, i simply tweak it until it does"; he only (partially) > changed that stance and relied a bit more on mdoc(7) and roff(7) > documentation some time after the advent of mandoc. I perceive a self-defeating cycle. Real Programmers write code, not documentation, so bug reports against a project regarding its incorrect or deficient documentation get either ignored or contemptuously rejected. Thus people learn that there's no point reporting such things. > Most authors of man(7) code generators clearly do the same. > The end result is that not only documented behaviour > matters, but it can easily happen that manual pages rely > on undocumented behaviour. It is a risk, yes. Fortunately for _groff_ development, few authors of man page generators explore as aggressively as _mandoc_'s test suite. > 3. Mandoc(1) does not only aim for compatibilty with groff > documentation, but for compatibility with groff output, > aka bug-for-bug compatibility. That requires some testing > of undocumented features. I think I understand that choice, but you must remember that not all _groff_ behavior is correct even according to its own developers. One look at its ticket list in Savannah will tell you that. > 4. While groff(1) documentation is good, it is not complete > and detailed enough for guiding a reimplementation. I'd like for it to be, but I may never achieve that goal. It is years away, at least. > 5. The test suite is primarily a mandoc(1) test suite, that is, > its main purpose is to make sure that mandoc(1) output does > not accidentally change, not even in cases where groff > behaviour is undocumented. Consequently, the test suite aims > to exercise all code paths in mandoc(1) and all possible > combinations of input tokens related to the feature being > tested, with no regard to which combinations are documented. > While test suite completeness is hard to define and hard to > achieve, it is a worthy goal and should not be confused > with overtesting. I remind you that I'm not the person applying the term "overtesting" to _mandoc_'s test suite. I came close, maybe, but not quite. My advice is, upon exercising undefined behavior, make as few assumptions as possible about how GNU _troff_ will recover from the error state. Unfortunately if you were to challenge me to articulate the minimal set of assumptions you *should* make, I would say: "Upon encountering invalid input on a control line, ignore the rest of the "logical" input line." ...and then flounder because on text lines, all we have to deal with are escape sequences, and thus the topic of the present ticket. I don't have a rule I can state regarding what the formatter should do when an escape sequence goes off the rails. I don't expect to have an easy rule any time soon, because some escape sequences do one thing (one "primitive" operation, in some sense), and some can do many. `\b` and `\o` each exist for the purpose of typesetting *multiple* characters. A drawing command `\D` can produce polylines. `\Z` can enclose a series of other escape sequences, including other occurrences of itself. These observations render even the initial one about control lines incomplete, because escape sequences can break the rule I just articulated about control lines. > When the test suite fails on the mandoc side, you generally > don't hear about it, i simply fix it myself. When it fails on > the groff side, i try to classify it. The reason may be: > > a) A groff regression that must be fixed in the groff port > and reported upstream. Yes, and I certainly want to hear about this. It's just that not all "regressions" in that sense are regressions in _groff_ per se. > b) A minor groff behaviour change that doesn't matter much > either way and that mandoc(1) can simply follow. > c) A bug fix in groff that mandoc(1) should follow. > d) A feature improvement in groff that mandoc(1) should follow. > > I don't think adding the following class would make much sense: > > e) Change in undefined behaviour, just delete the test. Agreed--don't delete it, but don't make assumptions about what parser state recovery looks like. > In mandoc(1), i definitely don't want accidental output changes > even for invalid input. Even for invalid input, i always want > to make sure that the new output makes more sense than the old > output. I think aiming for that would make sense in groff, too. It's a good principle, but may be difficult to apply. If I followed my basest instincts, the formatter would abort upon encountering any trouble. > I don't think the commit that caused this regression (or behaviour > change, if you want - but given that the behaviour change was > accidental rather than intentional, the term "regression" seems > fitting to me, no matter whether the behaviour is defined or not) > did not meet that standard. The _output_ of _your test suite_ regressed, certainly. That doesn't necessarily mean you discovered a defect in _groff_. You noted cases (c), and (d). > I do not see how it made the output > or the diagnostics better. Rather, both degraded - the output > lost text present in the input file, and the new warning message was > quite confusing and did not help the user in diagnosing the problem. That may be, but consider the effect of my proposed patch in comment #11. $ cat ATTIC/67372a.groff .ds v did \A\*vs $ ./build/test-groff -aww ATTIC/67372a.groff troff:ATTIC/67372a.groff:2: warning: identifier validation escape sequence expected a delimiter, got an escape character <beginning of page> dids $ ./build/test-groff -Caww ATTIC/67372a.groff <beginning of page> 1s $ ~/groff-1.22.4/bin/groff -a ATTIC/67372a.groff <beginning of page> 1s You'd certainly get "more text" than with _groff_ 1.22.4 or an AT&T-based _troff_. Better, it would be text you could grep the document for ("did", contrast with the "1" synthesized by escape sequence interpolation). > I find it quite surprising that the root cause for this fallout > turned out to be "a jjc@ feature that still isn't properly > documented nor unambiguously designed nor consistently implemented". > Had you asked me, i would hardly have thought that issues of > this kind still exist in groff. Speaking as a weary maintainer and old engineer, nothing like that surprises me. > Really surprising for groff > to still suffer from what can essentially be described as > a "paedriatic disease". This regression now looks like the > runny nose and the rash that unavoidably comes with such a > childhood illness. This is why I'm a fan of formal verification and rate languages like Ada and Haskell above C, C++, and the Bourne (or even POSIX) shell. Users of the latter, and I've been one of those for a while, develop overconfidence in their command of a program's (or script's) actual behavior. In my experience, the more you hear someone boasting of their using of a language that's "close to the metal", the less they understand the system and the less likely you are ever to see them studying a disassembly, let alone actually writing anything in the assembler. Maybe I've been unlucky. None of this takes anything away from James Clark's accomplishments. I think he did brilliantly well, and much better than I would have expected of myself, in producing usable software with the bucking bronco of pre-standard C++ as his means of transport. But that language and its ecosystem was, and some argue still is, a chaos engine. And over time, no matter how talented the programmer, the fact will tell. To conclude, it can be profitable to revisit our conclusions about correctness from prior principles. _______________________________________________________ Reply to this item at: <https://savannah.gnu.org/bugs/?67372> _______________________________________________ Message sent via Savannah https://savannah.gnu.org/
signature.asc
Description: PGP signature
