`\c`, mdoc(7), and man(7) extension macros (was: [PATCH 1/2] man*/: srcfix)

[G. Branden Robinson]

Hi Alex,

At 2023-10-25T21:38:59+0200, Alejandro Colomar wrote:
> On Wed, Oct 25, 2023 at 01:54:24PM -0500, G. Branden Robinson wrote:
> > diff --git a/man2/open.2 b/man2/open.2
> > index 4c921723c..6603dfdff 100644
> > --- a/man2/open.2
> > +++ b/man2/open.2
> > @@ -82,8 +82,13 @@ .SH DESCRIPTION
> >  to an entry in the process's table of open file descriptors.
> >  The file descriptor is used
> >  in subsequent system calls
> > -.RB ( read "(2), " write "(2), " lseek "(2), " fcntl (2),
> > -etc.) to refer to the open file.
> > +(\c
> 
> I'm going to disagree with Ingo with his claim that a macro that
> forces using \c is bad because it promotes bad style.  '(\c' doesn't
> look bad to me here.  Not more than having the leading punctuation as
> an Nth argument.

I disagree with Ingo on that point as well.

Saying why leads me to digress; I found myself writing down thoughts
about future man(7) development more concretely than I have to date.
(I'll return to this patch at the end.)  So, Ccing the groff list...

I think Ingo's perspective is strongly influenced by mdoc(7), the use of
which he strenuously advocates.  And mdoc _does_ manage to make `\c`
almost(?) totally unnecessary--at the cost of a weighty internal
recursive macro reprocessing system that no other *roff package is known
to implement.

(This is what that "parsed"/"callable" stuff in groff_mdoc(7) (and
mandoc_mdoc(7)) is all about.  Also, by "weighty", I mean it--back in
~1990, when mdoc was implemented, its documentation warned the reader of
its slowness.  Fortunately, on modern systems, the rendering latency
relative to man(7) is no longer noticeable.)

Even with performance considerations out of the picture, I think such a
system is a point against adoption of mdoc; one can observe that,
nowadays, both man(7) and mdoc require a person to acquire knowledge
that they will "never" transfer anywhere else, assuming no resurgence in
*roff popularity.  But the point is _even more true_ of mdoc than it is
of man(7), by dint of the recursive macro reprocessing feature and the
funny requirements that become necessary as a consequence (you have to
use `\&` in more places; you have to break up individual punctuation
characters when using them as macro arguments if you want them to work
right).  In other words, learning mdoc doesn't help you learn *roff all
that much.  Not as much as man(7) does.  And it doesn't help you learn
any other programming/documentation system anywhere.

Plus, in mdoc, there is a much larger lexicon of macros to learn in the
first place.  I count 115.

$ git grep '\.\(als\|de\) [A-Z][A-Za-z1][a-z]*' \
  tmac/doc.tmac tmac/mdoc/doc-!(nroff) | wc -l
115

Admittedly, a couple of those are, if not deprecated, unlikely to be
used in practice, because even after mdoc's documentation spent decades
asking its readers for help, no one could be found who clearly remembers
what they were supposed to do (`Me`, `Ot`).  So, call it 113 macros.

But Ingo added `Tg` to mandoc(1) mdoc--which groff doesn't support yet--
so make that 114.

man(7), in groff 1.23.0, has 30.  2 of those (`LP`, `PP`) are redundant
synonyms for `P` that I'd happily kill off if the man page authors of
the world would let me.  Six others, the font alternation macros--`BI`,
`BR`, and friends--are schematic and so are relatively "light" on human
memory.  Several more come in pairs (`EX`/`EE`, `MT`/`ME`, `RS`/`RE`,
`SY`/`YS`, `UR`/`UE`).[1]  I confess that I do have plans to grow
man(7)'s macro repertoire for groff 1.24, but not by more than a few,
and only for purposes where we see concrete needs.[2]  Further, `SB` is
already deprecated in Git (it was never "standard" in the first place).

Granted, some of mdoc's macros are systematically named as well.  The
enclosure macros are probably the prime example.

groff_mdoc(7):
   Quote   Open   Close   Function  Result
   .Aq .Ao.Ac Angle Bracket Enclosure   
   .Bq .Bo.Bc Bracket Enclosure [string]
   .Brq.Bro   .BrcBrace Enclosure   {string}
   .Dq .Do.Dc Double Quote  “string”
   .Eq .Eo.Ec Enclose String (in XY)XstringY
   .Pq .Po.Pc Parenthesis Enclosure (string)
   .QlQuoted Literal“string” or string
   .Qq .Qo.Qc Straight Double Quote "string"
   .Sq .So.Sc Single Quote  ‘string’

None of this is to say that mdoc should go away.  I do not mirror
mandoc(1)'s initial ambition to completely kill off man(7).[3]  There's
room in the world for mdoc, and I think it can coexist with man(7).  But
I tire both of mdoc advocates' derogations of man(7), and of resistance
to development of the man(7) language so intransigent that it keeps the
macro package from rising to the challenge mdoc has presented to it,
particularly in areas that man(7) authors also complain about.  A
community with totalitarian con

Re: `\c`, mdoc(7), and man(7) extension macros (was: [PATCH 1/2] man*/: srcfix)

[Alejandro Colomar]

Hi Branden,

On Thu, Oct 26, 2023 at 07:58:35AM -0500, G. Branden Robinson wrote:
[...]
> 
> And mdoc _does_ manage to make `\c`
> almost(?) totally unnecessary--at the cost of a weighty internal
> recursive macro reprocessing system that no other *roff package is known
> to implement.
> 
> (This is what that "parsed"/"callable" stuff in groff_mdoc(7) (and
> mandoc_mdoc(7)) is all about.  Also, by "weighty", I mean it--back in
> ~1990, when mdoc was implemented, its documentation warned the reader of
> its slowness.  Fortunately, on modern systems, the rendering latency
> relative to man(7) is no longer noticeable.)
> 
> Even with performance considerations out of the picture, I think such a
> system is a point against adoption of mdoc; one can observe that,
> nowadays, both man(7) and mdoc require a person to acquire knowledge
> that they will "never" transfer anywhere else, assuming no resurgence in
> *roff popularity.  But the point is _even more true_ of mdoc than it is
> of man(7), by dint of the recursive macro reprocessing feature and the
> funny requirements that become necessary as a consequence (you have to
> use `\&` in more places; you have to break up individual punctuation
> characters when using them as macro arguments if you want them to work
> right).  In other words, learning mdoc doesn't help you learn *roff all
> that much.  Not as much as man(7) does.  And it doesn't help you learn
> any other programming/documentation system anywhere.
> 
> Plus, in mdoc, there is a much larger lexicon of macros to learn in the
> first place.  I count 115.

Yup.  I have similar feelings about C++, BTW.  I have a hard time
understanding complex languages.  I prefer simpler languages.  Most
features can be implemented as library code, without complicating the
language.

[...]

> > 
> > For consistency with the above two cases, I think you should move that
> > (\c to a new line.  It also reduces the diff (semantic newlines any?)
> > :)
> 
> Well, okay.  I'll roll a v5.
> 
> By the way, even the diffless version of 2/2 (the actual `MR` migration)
> got canned by vger.  Even just the diff --stat blows past the 100,000
> byte limit.  But you should have a copy in your inbox, and the sed
> script is what does the real work of 2/2 anyway.

Yep, I have 2/2 locally.  Maybe reply to the mail CCing vger, keeping
only the commit message, so that readers of the list can get a notice of
what MR.sed is.

> 
> Regards,
> Branden
> 
> [1] It _would_ be nice if these all ended in `S` and `E`, respectively,
> for "start" and "end".  Alas, there is historical baggage here,
> which Doug McIlroy, author of the original man(7) macros and of the
> latter Ninth Edition Unix `EX`/`EE` extension, recently lamented.
> 
> https://lists.gnu.org/archive/html/groff/2023-09/msg00058.html
> 
> We _could_ fix all the outliers, save one, without collision.
> 
>   EX -> ES (example start)
>   MT -> MS (mail-to start)
>   UR -> US (URL start)
>   YS -> SE (synopsis end)

I always make the mistake of writing MS instead of MT, out of inertia of
having an 'S' for the start!  :D

The others have nicer mnemonics that work for me; I wouldn't change
them.  And well, just for changing MT, I wouldn't do it.

> 
> The oddball is `SY`.  We can't rename that to `SS`, which is already
> a subsection heading macro.  But we could rename _both_ synopsis
> macros.
> 
>   SY -> NS (syNopsis start)
>   YS -> NE (syNopsis end)

Hmmm, no; I don't like it.

> 
> I'm game for any of these reforms, if people think it's worth it.
> The old names can be kept for backward compatibility for as long as
> necessary (but de-documented).  I _assume_ there's too much inertia
> for this.

Regarding PP, LP, and P, what's the history of them?  Why do we have the
3?  I'm willing to reduce them to just one.


Cheers,
Alex

-- 



signature.asc
Description: PGP signature

Re: 1.23 prints some strange error

[G. Branden Robinson]

At 2023-10-25T16:20:27+0200, Walter Alejandro Iglesias wrote:
> What you did above is not the step by step way I posted to reproduce
> the bug.  Of course it won't be helpful if you overlook it.

You've already gotten what I would have thought to be a sufficient
explanation of the diagnostic messages you saw.

1.  GNU troff (the formatter program) doesn't accept UTF-8 input;
2.  Your list of hyphenation exceptions (`hw` requests) is formatted in
UTF-8;
3.  Your document is using `mso` rather than `so` requests to load the
list of hyphenation exceptions;
4.  The soelim(1) program does not operate on `mso` requests (nor should
it, in my opinion); therefore,
5.  Your input confuses the formatter, producing diagnostics.

Why these exact diagnostics?

Well, let's have a look at the first.

$ nroff -M. ./doc.tr
troff:./list.tr:1: error: expected ordinary or special character, got an 
escaped '%'
$ head -n 1 list.tr
.hw a-hí
$ hd list.tr
  2e 68 77 20 61 2d 68 c3  ad 0a 2e 68 77 20 61 2d  |.hw a-hhw a-|
0010  c3 b1 6f 0a 2e 68 77 20  c3 a1 72 2d 62 6f 6c 0a  |..o..hw ..r-bol.|
0020  2e 68 77 20 63 75 2d 62  72 c3 ad 2d 61 0a 2e 68  |.hw cu-br..-a..h|
0030  77 20 65 2d 74 c3 a9 2d  72 65 2d 6f 0a 2e 68 77  |w e-t..-re-o..hw|
0040  20 63 61 2d 6d 69 c3 b3  6e 0a 2e 68 77 20 c3 ba  | ca-mi..n..hw ..|
0050  2d 74 65 2d 72 6f 0a 2e  68 77 20 70 69 6e 2d 67  |-te-ro..hw pin-g|
0060  c3 bc 69 2d 6e 6f 0a  |..i-no.|
0067

GNU troff reads line 1 of list.tr, interpreting it as ISO Latin-1.  The
bytes of interest are therefore 0xc3 and 0xad.

C3 is "LATIN CAPITAL LETTER A WITH TILDE".

AD is "SOFT HYPHEN".

groff_char(7) explains what the formatter does with the latter.

   Eight‐bit encodings and Latin‐1 supplement
   ISO 646 is a seven‐bit code encoding 128 code points; eight‐bit
   codes are twice the size.  ISO 8859‐1 and code page 1047
   allocated the additional space to what Unicode calls “C1
   controls” (control characters) and the “Latin‐1 supplement”.  The
   C1 controls are neither printable nor usable as groff input.

   Two Latin‐1 supplement characters are handled specially on input.
   troff never produces them as output.

   NBSP   encodes a no‐break space; it is mapped to \~, the
  adjustable non‐breaking space escape sequence.

   SHYencodes a soft hyphen; it is mapped to \%, the hyphenation
  control escape sequence.

The formatter does not expect to see a hyphen control escape sequence
inside the definition of a hyphenation exception, and it complains if it
gets one.

That is why you got the error message you did.

That is why my advice is to either maintain files you `mso` in Latin-1
(or ASCII), or go ahead and maintain them in UTF-8, but as ".in" files
that your Makefile converts to input GNU troff will accept, using
preconv.

list.tr: list.tr.in
preconv -e utf-8 $< > $@

GNU troff does not reject code points A0-FF as invalid because they
aren't invalid; every single one might be found in a valid Latin-1
document.  The formatter _does_ reject code points 80-9F as input.  That
might not come up when inadvertently giving the formatter (valid) UTF-8
input, however; I haven't done the arithmetic, but it seems possible to
me that some or all of these would be treated as "overlong encodings" of
Basic Latin code points.

See, e.g., "Canonicalization of Non-Shortest Form UTF-8".

https://websec.github.io/unicode-security-guide/character-transformations/

As it happens, GNU troff uses the C1 Control block (U+0080..U+009F) for
internal purposes.  That is one of the reasons it's non-trivial to
covert it to understand UTF-8 natively, an outcome pretty much everyone
desires.

https://git.savannah.gnu.org/cgit/groff.git/tree/src/roff/troff/input.h?h=1.23.0

Regards,
Branden


signature.asc
Description: PGP signature

Why does man(7) have 3 paragraph macros for the same thing? (was: `\c`, mdoc(7), and man(7) extension macros)

[G. Branden Robinson]

At 2023-10-26T16:12:36+0200, Alejandro Colomar wrote:
> Regarding PP, LP, and P, what's the history of them?  Why do we have
> the 3?  I'm willing to reduce them to just one.

I invite Doug McIlroy to go on record, but my surmise is that they were
introduced as crutches for people already familiar with ms(7).

Doug's original man(7) (1979) didn't have `P`.  But Unix System III
added it in 1980, and 4.3BSD followed suit in 1986.  This information is
in groff_man(7).

In ms(7), `LP` sets a paragraph that is left-aligned, and `PP` sets one
with a first-line indentation.

In man(7), all (ordinary) paragraphs behave analogously to ms(7)'s `PP`.
A first-line indentation is never used.

My opinion is that one should use `P` exclusively for a few reasons.

1.  Its name is short and it is frequently used.  Huffman coding FTW.
2.  The existence of both `LP` and `PP` is difficult to rationalize to
anyone except a seasoned ms(7) author, of whom there are almost none
writing man(7) documents today.
3.  It's been around long enough that everything supports it.  `LP`/`PP`
have no measurable advantage in this respect.  Anyone running early
BSD in 2023 has more interesting things to do than the composition
of legacy man pages.

Incidentally, mm(7) would have already had a `P` macro as well by 1979.
Its call syntax is different (it interprets arguments; ms(7)'s and
man(7)'s "plain" paragraphing macros do not).  Maybe this, and/or some
friction between Bell Labs Research and the constantly reconfiguring
series of departments dedicated to deriving revenue from Unix (and
troff) commercially accounts for man(7)'s omission of it.

Nowadays, we can be confident that no loon is going to do something
crazy with mm's `P` macro such that it would trip up man(7) writers.



Regards,
Branden


signature.asc
Description: PGP signature

Re: Why does man(7) have 3 paragraph macros for the same thing? (was: `\c`, mdoc(7), and man(7) extension macros)

[Alejandro Colomar]

Hi Branden,

On Thu, Oct 26, 2023 at 09:51:40AM -0500, G. Branden Robinson wrote:
> At 2023-10-26T16:12:36+0200, Alejandro Colomar wrote:
> > Regarding PP, LP, and P, what's the history of them?  Why do we have
> > the 3?  I'm willing to reduce them to just one.
> 
> I invite Doug McIlroy to go on record, but my surmise is that they were
> introduced as crutches for people already familiar with ms(7).
> 
> Doug's original man(7) (1979) didn't have `P`.  But Unix System III
> added it in 1980, and 4.3BSD followed suit in 1986.  This information is
> in groff_man(7).

Was the original PP?

> 
> In ms(7), `LP` sets a paragraph that is left-aligned, and `PP` sets one
> with a first-line indentation.
> 
> In man(7), all (ordinary) paragraphs behave analogously to ms(7)'s `PP`.
> A first-line indentation is never used.
> 
> My opinion is that one should use `P` exclusively for a few reasons.
> 
> 1.  Its name is short and it is frequently used.  Huffman coding FTW.
> 2.  The existence of both `LP` and `PP` is difficult to rationalize to
> anyone except a seasoned ms(7) author, of whom there are almost none
> writing man(7) documents today.

Still, compatibility with ms(7) would make it slightly easier to
trasnfer learning from man(7) to ms(7), would one learn it.  I know many
other macros are incompatible in bad ways, but the less the better, no?

Cheers,
Alex

> 3.  It's been around long enough that everything supports it.  `LP`/`PP`
> have no measurable advantage in this respect.  Anyone running early
> BSD in 2023 has more interesting things to do than the composition
> of legacy man pages.
> 
> Incidentally, mm(7) would have already had a `P` macro as well by 1979.
> Its call syntax is different (it interprets arguments; ms(7)'s and
> man(7)'s "plain" paragraphing macros do not).  Maybe this, and/or some
> friction between Bell Labs Research and the constantly reconfiguring
> series of departments dedicated to deriving revenue from Unix (and
> troff) commercially accounts for man(7)'s omission of it.
> 
> Nowadays, we can be confident that no loon is going to do something
> crazy with mm's `P` macro such that it would trip up man(7) writers.
> 
> 
> 
> Regards,
> Branden



-- 



signature.asc
Description: PGP signature

Re: 1.23 prints some strange error

[Walter Alejandro Iglesias]

On Thu, Oct 26, 2023 at 09:35:07AM -0500, G. Branden Robinson wrote:
> At 2023-10-25T16:20:27+0200, Walter Alejandro Iglesias wrote:
> > What you did above is not the step by step way I posted to reproduce
> > the bug.  Of course it won't be helpful if you overlook it.
> 
> You've already gotten what I would have thought to be a sufficient
> explanation of the diagnostic messages you saw.
>
> [...]

Ah, now I understand where's the problem, as it happens to me, groff
1.22.4 also ingores what you're explaining.

-- 
Walter

Re: Why does man(7) have 3 paragraph macros for the same thing? (was: `\c`, mdoc(7), and man(7) extension macros)

[G. Branden Robinson]

At 2023-10-26T16:58:13+0200, Alejandro Colomar wrote:
> On Thu, Oct 26, 2023 at 09:51:40AM -0500, G. Branden Robinson wrote:
> > At 2023-10-26T16:12:36+0200, Alejandro Colomar wrote:
> > > Regarding PP, LP, and P, what's the history of them?  Why do we
> > > have the 3?  I'm willing to reduce them to just one.
> > 
> > I invite Doug McIlroy to go on record, but my surmise is that they
> > were introduced as crutches for people already familiar with ms(7).
> > 
> > Doug's original man(7) (1979) didn't have `P`.  But Unix System III
> > added it in 1980, and 4.3BSD followed suit in 1986.  This
> > information is in groff_man(7).
> 
> Was the original PP?

It had both `PP` and `LP`.  I reckon Doug figured ms(7) veterans would
have an unreformable habit of typing one or the other.

https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/man/man7/man.7

I'd have been sterner--and probably less fondly remembered by my peers.

(While I'm armchair quarterbacking Doug's plays of 45 years ago, oh, how
I wish he hadn't have used input traps for anything in man(7).[1])

> Still, compatibility with ms(7) would make it slightly easier to
> trasnfer learning from man(7) to ms(7), would one learn it.  I know
> many other macros are incompatible in bad ways, but the less the
> better, no?

That's true, but these days the knowledge transfer is, I submit, vastly
more likely to go the other way; that is, people will be exposed to
man(7) as their first roff macro language, and might decide to pick up
ms(7).

At that point, they'd have to learn that `LP` and `PP` do _different_
things.  I think it's actually better if they _don't_ have to unlearn
the "fact" (applicable only to man(7)) that they are exactly the same.

Better, I believe, to promote only `P` in man(7).  Anyone wanting to
pick up mm(7) will still enjoy some knowledge transfer.  Without
arguments, `P` in mm(7) "does what you mean".

(I will not elaborate here on what that means; see the groff_mm(7) man
page in groff 1.23.0 and please God not an earlier version.)

Regards,
Branden

[1] In practice, nearly no one took them up for any purpose except the
one place you _had_ to use them: `TP`.


signature.asc
Description: PGP signature

Re: 1.23 prints some strange error

[G. Branden Robinson]

Hi Walter,

At 2023-10-26T17:21:21+0200, Walter Alejandro Iglesias wrote:
> Ah, now I understand where's the problem, as it happens to me, groff
> 1.22.4 also ingores what you're explaining.

For me, the output doesn't change, but groff 1.22.4 doesn't emit a
diagnostic.

But that's not a surprise.  After 1.22.4 was released, I added a _ton_
of diagnostics when validity problems are encountered.  I have no idea
how many.  Dozens.  Changes in _behavior_ are documented in the "NEWS"
file.  If there was any change to formatting not documented there, I
would very much like to know about it.

I updated your doc.tr file to test this claim.

$ cat ./doc.tr
.mso list.tr
.nr i 49 -1
.while \n+i \{\
ahí
.\}
$ /usr/bin/nroff -v
GNU nroff (groff) version 1.22.4
$ /usr/bin/nroff -M. ./doc.tr | head -n 3
ahà  ahà  ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahÃ
ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà  ahà  ahÃ
ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahÃ
$ nroff -v
GNU nroff (groff) version 1.23.0
GNU groff version 1.23.0
Copyright (C) 2022 Free Software Foundation, Inc.
GNU groff comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of groff and its subprograms
under the terms of the GNU General Public License.
For more information about these matters, see the file
named COPYING.

called subprograms:

GNU grotty (groff) version 1.23.0
GNU troff (groff) version 1.23.0
$ nroff -M. ./doc.tr | head -n 3
troff:./list.tr:1: error: expected ordinary or special character, got an 
escaped '%'
troff:./list.tr:4: error: expected ordinary or special character, got an 
escaped '%'
ahà  ahà  ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahÃ
ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà  ahà  ahÃ
ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahà ahÃ

Regards,
Branden


signature.asc
Description: PGP signature

Re: Why does man(7) have 3 paragraph macros for the same thing? (was: `\c`, mdoc(7), and man(7) extension macros)

[G. Branden Robinson]

[self-correcting follow-up]

At 2023-10-26T09:51:42-0500, G. Branden Robinson wrote:
> In ms(7), `LP` sets a paragraph that is left-aligned, and `PP` sets
> one with a first-line indentation.
> 
> In man(7), all (ordinary) paragraphs behave analogously to ms(7)'s
> `PP`.  A first-line indentation is never used.

Sorry, brain fart here.

In man(7), all (ordinary) paragraphs behave analogously to ms(7)'s `LP`.
A first-line indentation is never used.

Regards,
Branden


signature.asc
Description: PGP signature

Re: mandoc -man -Thtml bug: inconsistent vertical space before .TP

[Ingo Schwarze]

Hi Branden and Alejandro,

G. Branden Robinson wrote on Tue, Oct 24, 2023 at 04:54:21AM -0500:
> At 2023-10-24T02:13:34+0200, Ingo Schwarze wrote:

>>  5. On top of all that, i have a hard time to think of any macro
>> that has a more wicked failure mode than .TQ in case the
>> formatter does not support it.  The output visually looks
>> perfectly fine, and the reader gets no hint that the *most
>> important* information is missing.

> I think you're getting carried away here.  `TQ` _takes no arguments_,
> so if a formatter completely ignores it, no text goes missing.

That is completely true, and my point 5 is completely bogus.  This is
embarrassing, in particular since it also tainted the commit message.
I'm not completely sure how the wrong idea entered my head, possibly by
confusing .TP and .IP, which i tend to confuse regularly, even though
the mnemonics isn't actually bad (.TP is the one that can *only*
be reasonably used for lists with a tag, i.e. the one with mandatory
next-line syntax, whereas .IP is the one that can be used for any
indented paragraph, even without a tag).  Not an excuse of course,
i should have looked more closely and not jumped to wrong conclusions.

> Description
>abra   cadabra Newport News.

Yes, that's good enough as a fallback for a few rarely used formatters.

> I think you may have mixed in your own fulminations against the
> new `MR` macro (whose presence in groff is my doing) with your
> apprehension about `TQ`, which predates my participation in groff
> development by several years.

That's another possible factor which might have contributed to my
massive gaffe, yes.

> I'm a bit dubious of `TQ` myself, but it does fulfill, partially,
> a need that mdoc(7) meets with `.Bl -compact`.

Yes, i fully agree with that.
My points 1. to 4. remain valid, but that only means .TQ should not
be used excessively.  The fact that my point 5 was bogus means
that avoiding it almost completely is not required.
Indeed, reasonable use cases seem the same as for .Bl -compact
with seletive .Pp.  The semantic downside of having a list
entry with an empty body instead of having an item header consisting
of two lines is also the same in both languages.  Then again, while
treating an empty item body as an indication for semantic formatters
that the following body has two tags is a burden on formatters, it's
probably not too heavy to bear, so inventing new syntax to avoid that
burden is likely not reasonable.

> If we were to add my proposed list macros, LS/LE to man(7), we could
> de-document `TQ`, as it would be redundant with stacked, bodyless `TP`s
> inside `LS`/`LE` brackets.
> 
> https://lists.gnu.org/archive/html/groff/2022-12/msg00075.html
> 
> I haven't explicitly made the connection to HTML before,

Well, when designing a new language from scratch, you should
always consider prior art, to reduce the risks of reinventing the
wheel, introducing new design mistakes, and leaving design gaps.
In particular, when designing a markup language for documentation, i
consider it critical to carefully compare the design to HTML, LaTeX,
and mdoc(7) before making final decisions, and there may be a few
more that might also be worth looking at for comparison.  Looking at
DocBook is likely *not* a good idea though simply because its design is
so atrocious in so many respects that you will waste massive amounts
of time without learning anything, excpt maybe what not to do.
That doesn't mean the new design must follow the existing languages,
but not even considering HTML 5 when designing a markup language
feels like straightforward negligence to me.  :-(

> but it's pretty easy to imagine this mapping.
> 
> LS -> 
> TP ->  ... 
> LE -> 
> 
> Is that a tradeoff you'd be willing to make?

Well, *if* you really want to totally redesign the very foundations
of man(7) and change it from almost presentation-only and almost
in-line-macro only to the totally different paradigms of semantic
markup and block oriented, that is definitely one among the many task
involved in redesigning.

Yes, structural markup of a list requires saying where the list
starts, where the list ends, and where each item begins.  So that
part of the design of .LS feels right.  If i understand correctly,
the .LS macro will not accept text arguments, which is also good.
The naming seems fair, too.  I did *not* review the proposal though,
so there may be downsides that i am unaware of.  All i'm saying is
that these three points look good, and that the man(7) language indeed
has one of its major weaknesses in this area.

> They could always have done what `TQ` itself does:
> 
> .br
> .ns
> 
> and then
> 
> .TP
> again as usual.
> 
> But that's two low-level requests instead of zero, and one of those
> might take some explaining since it exercises a formatter feature that
> only relatively advanced *roff users ever fool with.

I agree that .br+.ns is not better than .TQ in a man(7) page.

The following probably wouldn'

Re: Why does man(7) have 3 paragraph macros for the same thing?

[Ingo Schwarze]

Hi Branden and Alejandro,

G. Branden Robinson wrote on Thu, Oct 26, 2023 at 10:28:13AM -0500:
> At 2023-10-26T16:58:13+0200, Alejandro Colomar wrote:
>> On Thu, Oct 26, 2023 at 09:51:40AM -0500, G. Branden Robinson wrote:
>>> At 2023-10-26T16:12:36+0200, Alejandro Colomar wrote:

 Regarding PP, LP, and P, what's the history of them?  Why do we
 have the 3?  I'm willing to reduce them to just one.

>>> Doug's original man(7) (1979) didn't have `P`.  But Unix System III
>>> added it in 1980, and 4.3BSD followed suit in 1986.  This
>>> information is in groff_man(7).

>> Was the original PP?

> It had both `PP` and `LP`.

>> Still, compatibility with ms(7) would make it slightly easier to
>> trasnfer learning from man(7) to ms(7), would one learn it.  I know
>> many other macros are incompatible in bad ways, but the less the
>> better, no?

> That's true, but these days the knowledge transfer is, I submit, vastly
> more likely to go the other way; that is, people will be exposed to
> man(7) as their first roff macro language, and might decide to pick up
> ms(7).
> 
> At that point, they'd have to learn that `LP` and `PP` do _different_
> things.  I think it's actually better if they _don't_ have to unlearn
> the "fact" (applicable only to man(7)) that they are exactly the same.
> 
> Better, I believe, to promote only `P` in man(7).  Anyone wanting to
> pick up mm(7) will still enjoy some knowledge transfer.  Without
> arguments, `P` in mm(7) "does what you mean".

I consider this a bikeshed discussion.

Given that Branden apparently wants to
 * promote .P and deprecate .PP
 * i don't want mandoc_man(7) to gratuitiously spread any more bad
   man(7) style advice than is unavoidable by the fundamental decision
   of declaring the whole man(7) language as obsolete,
i briefly considered changing mandoc_man(7).

Currently it says:

  PP  Begin an undecorated paragraph.  The scope of a paragraph is closed
  by a subsequent paragraph, sub-section, section, or end of file.
  The saved paragraph left-margin width is reset to the default.

  LP  A synonym for PP.

  P   This synonym for PP is an AT&T System III UNIX extension later
  adopted by 4.3BSD.

and it declares LP and P deprecated by including only PP in the
MACRO OVERVIEW.

All the arguments feel weak in either direction:

 * In theory, .PP is more portable than .P, but that is extremely
   unlikely to ever matter in practice.
 * As seen above, the similarities and subtle differences
   when comparing to ms(7) can be employed as arguments in either
   direction.
 * The arguably more important similarity that HTML defines a 
   but not a  element can be regarded as a learning aid,
   but it's still a weak argument because HTML and roff(7) are
   very different domains and not similar in most other respects.
 * The similarity of .P and  can also be turned around to be
   levied as an argument for .PP:  .P and  are *very different*
   in so far as  is a block element, whereas .P is an in-line
   macro that cannot participate in block nesting.  In particular,
   it can neither nest inside a list item, nor can anything be
   contained inside a .P syntax tree node.  In contrast to ,
   .P does not represent a *paragraph*, but only a paragraph *break*.
 * .PP is more similar to mdoc(7) .Pp.  Again, a weak argument because
   macro naming is totally different in both languages even in most
   of the few cases where functionality matches, with the exception
   of only .SH and .SS.

Consequently, i tend to leave mandoc_man(7) just as it is and not
repaint the bikeshed.  That way, the original .PP macro - with which
nothing is really wrong, except for the fundamental design mistake
of not being a block macro, a mistake it shares with mdoc(7) .Pp -
gets the full description, while the slighly younger .P gets the
compat info, even though that now is only of historical but not
of practical interest.  Maybe still nice to keep both apart - gee,
yet another weak argument.

If, for some reason, you feel strongly about it and think it is
important which one to promote, it might be possible to convince me to
deprecate .PP and list .P as the non-deprecated form even though it
is theoretically less portable.  I must admit i don't particularly
like the idea, though.  It feels like taking a gratuitious risk,
which does not feel ideal even if both the magnitude of the risk
and the benefit reaped are almost exactly zero.

Yours,
  Ingo

Re: Why does man(7) have 3 paragraph macros for the same thing? (was: `\c`, mdoc(7), and man(7) extension macros)

[Alejandro Colomar]

Hi Branden,

On Thu, Oct 26, 2023 at 11:09:23AM -0500, G. Branden Robinson wrote:
> [self-correcting follow-up]
> 
> At 2023-10-26T09:51:42-0500, G. Branden Robinson wrote:
> > In ms(7), `LP` sets a paragraph that is left-aligned, and `PP` sets
> > one with a first-line indentation.
> > 
> > In man(7), all (ordinary) paragraphs behave analogously to ms(7)'s
> > `PP`.  A first-line indentation is never used.
> 
> Sorry, brain fart here.
> 
> In man(7), all (ordinary) paragraphs behave analogously to ms(7)'s `LP`.
> A first-line indentation is never used.

Hmm, then, since I don't like that name, either of PP and P sound good.
Being shorter, P seems the best one.

I'll probably change that soon.

Cheers,
Alex

-- 



signature.asc
Description: PGP signature

Re: Why does man(7) have 3 paragraph macros for the same thing? (was: `\c`, mdoc(7), and man(7) extension macros)

[Alejandro Colomar]

Hi Branden,

On Thu, Oct 26, 2023 at 10:28:13AM -0500, G. Branden Robinson wrote:
> At 2023-10-26T16:58:13+0200, Alejandro Colomar wrote:
> > On Thu, Oct 26, 2023 at 09:51:40AM -0500, G. Branden Robinson wrote:
> > > At 2023-10-26T16:12:36+0200, Alejandro Colomar wrote:
> > > > Regarding PP, LP, and P, what's the history of them?  Why do we
> > > > have the 3?  I'm willing to reduce them to just one.
> > > 
> > > I invite Doug McIlroy to go on record, but my surmise is that they
> > > were introduced as crutches for people already familiar with ms(7).
> > > 
> > > Doug's original man(7) (1979) didn't have `P`.  But Unix System III
> > > added it in 1980, and 4.3BSD followed suit in 1986.  This
> > > information is in groff_man(7).
> > 
> > Was the original PP?
> 
> It had both `PP` and `LP`.  I reckon Doug figured ms(7) veterans would
> have an unreformable habit of typing one or the other.
> 
> https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/man/man7/man.7
> 
> I'd have been sterner--and probably less fondly remembered by my peers.
> 
> (While I'm armchair quarterbacking Doug's plays of 45 years ago, oh, how
> I wish he hadn't have used input traps for anything in man(7).[1])
> 
> > Still, compatibility with ms(7) would make it slightly easier to
> > trasnfer learning from man(7) to ms(7), would one learn it.  I know
> > many other macros are incompatible in bad ways, but the less the
> > better, no?
> 
> That's true, but these days the knowledge transfer is, I submit, vastly
> more likely to go the other way; that is, people will be exposed to
> man(7) as their first roff macro language, and might decide to pick up
> ms(7).
> 
> At that point, they'd have to learn that `LP` and `PP` do _different_
> things.  I think it's actually better if they _don't_ have to unlearn
> the "fact" (applicable only to man(7)) that they are exactly the same.
> 
> Better, I believe, to promote only `P` in man(7).  Anyone wanting to
> pick up mm(7) will still enjoy some knowledge transfer.  Without
> arguments, `P` in mm(7) "does what you mean".

Yep; since PP is different in the languages, not using it here will be
better.  So, I'll soon s/PP/P/

Cheers,
Alex

> 
> (I will not elaborate here on what that means; see the groff_mm(7) man
> page in groff 1.23.0 and please God not an earlier version.)
> 
> Regards,
> Branden
> 
> [1] In practice, nearly no one took them up for any purpose except the
> one place you _had_ to use them: `TP`.



-- 



signature.asc
Description: PGP signature

Re: Why does man(7) have 3 paragraph macros for the same thing?

[Alejandro Colomar]

Hi Ingo,

On Thu, Oct 26, 2023 at 07:52:13PM +0200, Ingo Schwarze wrote:
> 
> I consider this a bikeshed discussion.

Sure.  But someone has to design the bike parkings.  I find a lot awful
bike parkings that harm bike's wheels, and have to park it in a sign or
tree nearby.

> 
> Given that Branden apparently wants to
>  * promote .P and deprecate .PP
>  * i don't want mandoc_man(7) to gratuitiously spread any more bad
>man(7) style advice than is unavoidable by the fundamental decision
>of declaring the whole man(7) language as obsolete,
> i briefly considered changing mandoc_man(7).
> 
> Currently it says:
> 
>   PP  Begin an undecorated paragraph.  The scope of a paragraph is closed
>   by a subsequent paragraph, sub-section, section, or end of file.
>   The saved paragraph left-margin width is reset to the default.
> 
>   LP  A synonym for PP.
> 
>   P   This synonym for PP is an AT&T System III UNIX extension later
>   adopted by 4.3BSD.
> 
> and it declares LP and P deprecated by including only PP in the
> MACRO OVERVIEW.
> 
> All the arguments feel weak in either direction:
> 
>  * In theory, .PP is more portable than .P, but that is extremely
>unlikely to ever matter in practice.
>  * As seen above, the similarities and subtle differences
>when comparing to ms(7) can be employed as arguments in either
>direction.
>  * The arguably more important similarity that HTML defines a 
>but not a  element can be regarded as a learning aid,
>but it's still a weak argument because HTML and roff(7) are
>very different domains and not similar in most other respects.
>  * The similarity of .P and  can also be turned around to be
>levied as an argument for .PP:  .P and  are *very different*
>in so far as  is a block element, whereas .P is an in-line
>macro that cannot participate in block nesting.  In particular,
>it can neither nest inside a list item, nor can anything be
>contained inside a .P syntax tree node.  In contrast to ,
>.P does not represent a *paragraph*, but only a paragraph *break*.
>  * .PP is more similar to mdoc(7) .Pp.  Again, a weak argument because
>macro naming is totally different in both languages even in most
>of the few cases where functionality matches, with the exception
>of only .SH and .SS.
> 
> Consequently, i tend to leave mandoc_man(7) just as it is and not
> repaint the bikeshed.  That way, the original .PP macro - with which
> nothing is really wrong, except for the fundamental design mistake
> of not being a block macro, a mistake it shares with mdoc(7) .Pp -
> gets the full description, while the slighly younger .P gets the
> compat info, even though that now is only of historical but not
> of practical interest.  Maybe still nice to keep both apart - gee,
> yet another weak argument.
> 
> If, for some reason, you feel strongly about it and think it is
> important which one to promote, it might be possible to convince me to
> deprecate .PP and list .P as the non-deprecated form even though it
> is theoretically less portable.  I must admit i don't particularly
> like the idea, though.  It feels like taking a gratuitious risk,
> which does not feel ideal even if both the magnitude of the risk
> and the benefit reaped are almost exactly zero.

I don't think there's any urgent need to change mandoc_man(7), since
good quality man(7) pages should not even read that page.  I see it as
a quick guide if you're in a mandoc(1) system and need to fix a man(7)
bug or something.  If you're going to write new man(7) pages, you
probably want to read groff_man(7).

But I think having 3 ways of spelling PP is bad, and I think deprecating
at least LP, and possibly one of P or PP would be a good move.

For making sure pages are fixed, we could an a warning that gets
triggered always, so that projects have time to catch the change.

As for chosing P or PP: I don't mind very much which, but P seems
slightly better.  Since both are relatively widespread, and I can help
turn the balance in favour of any of them, I'll side with groff(1)
using and recommending P.  But yeah, it's a very arbitrary decission
between P and PP.

Cheers,
Alex

> 
> Yours,
>   Ingo

-- 



signature.asc
Description: PGP signature

Re: mandoc -man -Thtml bug: inconsistent vertical space before .TP

[Alejandro Colomar]

On Thu, Oct 26, 2023 at 06:37:58PM +0200, Ingo Schwarze wrote:
> That is completely true, and my point 5 is completely bogus.  This is
> embarrassing, in particular since it also tainted the commit message.
> I'm not completely sure how the wrong idea entered my head, possibly by
> confusing .TP and .IP, which i tend to confuse regularly, even though
> the mnemonics isn't actually bad (.TP is the one that can *only*
> be reasonably used for lists with a tag, i.e. the one with mandatory
> next-line syntax, whereas .IP is the one that can be used for any
> indented paragraph, even without a tag).  Not an excuse of course,
> i should have looked more closely and not jumped to wrong conclusions.

No problem.  You're not a man(7) expert, as well as I'm no mdoc(7)
expert, so we can very well be off some days.  :)

> My points 1. to 4. remain valid, but that only means .TQ should not
> be used excessively.  The fact that my point 5 was bogus means
> that avoiding it almost completely is not required.
> Indeed, reasonable use cases seem the same as for .Bl -compact
> with seletive .Pp.  The semantic downside of having a list
> entry with an empty body instead of having an item header consisting
> of two lines is also the same in both languages.  Then again, while
> treating an empty item body as an indication for semantic formatters
> that the following body has two tags is a burden on formatters, it's
> probably not too heavy to bear, so inventing new syntax to avoid that
> burden is likely not reasonable.

I guess your opinion about the recent patch where I changed commas into
2-tag paragraphs is not so bad now, right?

Did you have a chance to check the patch that I'm about to push where I
reorder the options to put short options in the second tag, to make it
more appealing visually (less quasi-blank lines)?

I made a mistake when sending the patch, so I only sent it to the list,
forgetting to CC you, and bounced it to you afterwards, but maybe spam
filters got it, since you're not in the To or Cc list.  Anyway, you have
it here: .

> > Some Linux kernel hackers and other, ehrm, "stakeholders" might take
> > exception to your implication that the Linux man-pages endeavor is a GNU
> > project; it is not.  (It _does_ document a great deal of the GNU C
> > library nevertheless; this is the product of Info champion Richard
> > Stallman's immovable object meeting experienced Unix and GNU/Linux
> > developers' irresistible force demanding man pages.)
> 
> Oops, right.  Groff being a GNU project and groff on the one hand and
> Kerrisk + Colomar on the other hand cooperating so closely (which is
> also a very reasonable thing to do) i fell into another trap here.
> Sorry for that.

I may have contributed to the confusion, by changing the title of the
releases from

man-pages-5.13.tar.gz - man pages for Linux

to

man-pages-6.01 - manual pages for GNU/Linux

And the book says on the first page:

GNU/Linux
The man-pages book

The reason for that is that the project was too tied to Linux, but half
of it documents GNU, so it also felt a bit insulting to GNU (IMO) to not
even mention it.  So I added it.  I hope that helps get more
contributions from glibc programmers.  :)


Cheers,
Alex

> 
> > But that's just a nit about the commit message.  It's well down the list
> > of wrong things OpenBSD developers have said about GNU.  ;-)
> 
> Heh.  :-/
> 
> Misinformation is never good, no matter where it comes from.
> 
> Yours,
>   Ingo

-- 



signature.asc
Description: PGP signature

Re: `\c`, mdoc(7), and man(7) extension macros (was: [PATCH 1/2] man*/: srcfix)

[Ingo Schwarze]

Hi Branden and Alejandro,

G. Branden Robinson wrote on Thu, Oct 26, 2023 at 07:58:35AM -0500:
> At 2023-10-25T21:38:59+0200, Alejandro Colomar wrote:
>> On Wed, Oct 25, 2023 at 01:54:24PM -0500, G. Branden Robinson wrote:

>>> diff --git a/man2/open.2 b/man2/open.2
>>> index 4c921723c..6603dfdff 100644
>>> --- a/man2/open.2
>>> +++ b/man2/open.2
>>> @@ -82,8 +82,13 @@ .SH DESCRIPTION
>>>  to an entry in the process's table of open file descriptors.
>>>  The file descriptor is used
>>>  in subsequent system calls
>>> -.RB ( read "(2), " write "(2), " lseek "(2), " fcntl (2),
>>> -etc.) to refer to the open file.
>>> +(\c

>> I'm going to disagree with Ingo with his claim that a macro that
>> forces using \c is bad because it promotes bad style.  '(\c' doesn't
>> look bad to me here.  Not more than having the leading punctuation as
>> an Nth argument.

> I disagree with Ingo on that point as well.
> 
> Saying why leads me to digress; I found myself writing down thoughts
> about future man(7) development more concretely than I have to date.
> (I'll return to this patch at the end.)  So, Ccing the groff list...
> 
> I think Ingo's perspective is strongly influenced by mdoc(7), the use of
> which he strenuously advocates.  And mdoc _does_ manage to make `\c`
> almost(?) totally unnecessary--

I think the sentence is accurate even without the "almost".
I don't recall ever using \c in any mdoc(7) page, and not even
seeing it used by others, and i cannot think of any reason why
anyone should ever want to use it.

> at the cost of a weighty internal recursive macro reprocessing system
> that no other *roff package is known to implement.
> 
> (This is what that "parsed"/"callable" stuff in groff_mdoc(7) (and
> mandoc_mdoc(7)) is all about.  Also, by "weighty", I mean it--back in
> ~1990, when mdoc was implemented, its documentation warned the reader of
> its slowness.  Fortunately, on modern systems, the rendering latency
> relative to man(7) is no longer noticeable.)

This all seems accurate.  With groff or any other full roff(7)
implementation, parsing mdoc(7) is still significantly slower
than parsing man(7) (when regarding the same amount of text output).
Consequently, while mandoc(1) is significantly faster than groff for
both mdoc(7) and man(7), the speed benefit is *much* more pronounced
for mdoc(7) than for man(7).

Your argument that nowadays, sufficient computing power is usually
available such that these performance differences typically no longer
matter for interactive use also makes sense.

> Even with performance considerations out of the picture, I think such a
> system is a point against adoption of mdoc; one can observe that,
> nowadays, both man(7) and mdoc require a person to acquire knowledge
> that they will "never" transfer anywhere else, assuming no resurgence in
> *roff popularity.

Granted.  Then again, trying to find such aspects, i actually found
fewer than expected.  Here are some examples of aspects common to
both languages that aren't actually that unusual:

 * The '.' request/macro line marker is not that different from
   markers used for embedding in-band control information into
   various protocols and encoding schemes.
 * The same applies to the `\` character introducing escape sequences.
 * While \" is a very unusual way of introducing comments,
   it is used in the same way as comment markers in most other
   line-oriented languages.
 * Unescaped whitespace (SPACE and TAB) has syntactical meaning
   and needs escaping in many languages.
 * Protected whitespace ("\ ", \~) exists in many languages.
 * While some of the names of scaling units may be unusual,
   the practice of appending units to numbers is fundamental
   in all sciences.  Besides, scaling units follow traditions
   of classical typography and arguably aren't obsolete even in
   the digital age.
 * Quoting arguments containing whitespace in "" is widespread
   in many languages.
 * Backslash-newline for input line continuation is widespread
   among line-oriented languages, too.
 * Having multiple glyphs for various kinds of hyphens and dashes
   is ubiquitious in typography.
 * Having to pay attention to quotes and accents and their proper
   encoding in input files is ubiquitious in typography, too.

Roff idiosyncracies that are common to mdoc(7) and man(7) and
unlikely to help anywhere else include:

 * Arguably the most unusual feature is \& and its various use cases.
 * Escaping '\' requires \[rs] (GNU syntax) or \e (portable syntax),
   and \\ has a totally different and rather complicated meaning.
 * The same goes for having to use "\&." rather than "\.".
 * That totally blank lines are not ignored in most contexts
   but more or less equivalent to .sp is certainly surprising.
 * Escaping " inside macro arguments by doubling it is quite unusual.

But the most scary idiosyncrasies of roff(7) syntax and semantics
appear only in low-level roff(7) and do not haunt manual page
authors, for example:

 * conditiona

Re: `\c`, mdoc(7), and man(7) extension macros (was: [PATCH 1/2] man*/: srcfix)

[Alejandro Colomar]

Hi Ingo,

[...ingoring the battle in which I don't want to participate...]
[My position here is that man(7) has problems, but I fear that jumping
to mdoc(7) *may* be worse; I can't know before jumping, which is a
problem.  Maybe if some pages are added written in mdoc(7), I get used
to it and start to like it, but until that happens, I'm sticking to
man(7).  For now, I prepared the repo to welcome mdoc(7) pages.]

But my mail was for the following.  I'd like to comment on this
comparison:

On Fri, Oct 27, 2023 at 01:27:53AM +0200, Ingo Schwarze wrote:
> 
> Active macros, same functionality in mdoc(7) and man(7):
>   .Dd/.Dt/.Os   .TH
>   .Sh   .SH
>   .Ss   .SS
>   .Xr   .MR
>   .Pp   .PP/.P
>   .Bd -literal  .EX/.EE
>   .Bl -bullet   .IP/.TP/.TQ   # -dash/-hyphen is almost identical

My guidelines in man-pages(7) recommend that you only use .IP \[bu]
for bullets (or \(bu if you want more portability).  I would remove TP
and TQ from there.

>   .Bl -column   tbl(1)
>   .Bl -enum .IP/.TP/.TQ

Similarly, this would be .IP [1] or .IP (1), but never TP nor TQ.

>   .Bl -tag  .IP/.TP/.TQ

And for this one, I only recommend TP and TQ (TQ as a Tag Qontinuation
--it sounds better in my head, where I don't see it written :p--).  I'd
remove IP here.

>   .Pf/.Ns   .BI/.IB \c
>   .Lk   .UR/.UE
>   .Mt   .MT/.ME
>   .Em   .I
>   .Sy   .B
>   .No   .BR/.IR/.RB/.RI

Cheers,
Alex

-- 



signature.asc
Description: PGP signature