Re: [Groff] Mission statement
Hi Peter, > Serving PDF manuals from the web has drawbacks, possibly serious ones: > it assumes an installed PDF reader, and there's significant latency > involved with firing one up. I think the first assumption is reasonable these days, and the second I only notice if it's Adobe Reader. > [Deri's] demonstration is excellent. Yes. It would be nice to have a ¶ by some of the destinations, e.g. "BUGS" on the left of the line with ¶ on the right, with the aim of doing a Copy Link Location for passing on to others? .../groff.pdf#BUGS Cheers, Ralph.
Re: [Groff] Mission statement
Hi, James K. Lowden wrote on Tue, Mar 18, 2014 at 12:03:33AM -0400: > On Mon, 17 Mar 2014 21:49:49 -0400 Peter Schaffter wrote: >> Serving PDF manuals from the web has drawbacks, possibly >> serious ones: it assumes an installed PDF reader, and there's >> significant latency involved with firing one up. Exactly. > In 2014, is there a single web browser remaining not configured to > display PDFs? Yes, mine for example. I have never installed any PDF plugin to any browser on any of my machines, and i only very rarely set up a browser to spawn an external PDF reader; and if so, only a very basic one, certainly not implementing clickable hyperlinks. Almost always, i'm working with the "offer download window" option for PDF content, which means that i mostly ignore all PDF content on the web except in very special cases where i'm *really* interested in a specific document *and* somewhat trust the source. The attack surface of the browser is already much larger than i'm confortable with. I'm certainly not going to expand it yet more by accepting PDF documents to my machine. Security-wise, PDF is one of the most dangerous file formats, nowadays. > Is the latency significant, really? For an interactive application, > a second or two matters. For documentation that takes time to read > anyway, I'm not sure it's that important. For looking up an option, it matters; for reading half a manual, it does not. So i wouldn't configure man(1) to run groff -mandoc -Tps /usr/share/man/man1/ksh.1 | gv - it causes noticable delay even on a reasonably modern machine. Even though mandoc -Tps /usr/share/man/man1/ksh.1 | gv - would be fast enough, i don't use it because less(1) lets me search the content. Yours, Ingo
Re: [Groff] Mission statement, second draft
Am 17.03.2014 22:44, schrieb Peter Schaffter: ... On the subject of implementing Heirloom troff's paragraph-at-once formatting and associated goodies, I wrote Gunnar about it. Here's what he had to say: "Sorry, but I haven't done anything related to those topics for several years. I've never looked much into the groff code. From what I remember, the fact that it was in C++ alone made it so different though that there were few if any similarities at a superficial glance. Implementing paragraph based line breaking took me several weeks back when I was intimately familiar with the surrounding code. The algorithm itself is complex and takes some time to understand. And then there is the task of rewriting large portions of an existing environment that assumes line based formatting." Which is by way of saying it's going to be a big job, and we *must*, as a group, figure out how to attract programmers interested in tackling it. Line-by-line formatting is, IMO, the single biggest stumbling block to more widespread adoption of groff as a typesetting system. ... GROFF MISSION STATEMENT, 2014, 2nd draft ... > Backend The biggest challenge facing groff is the implementation of paragraph-at-once formatting based on the Knuth-Plass algorithm. Already present in Heirloom troff, this is a high-priority next step in groff's evolution, along with the addition of typesetting features modelled after Heirloom troff. ... Peter, it would be fine if you could provide an example text, formatted both with groff and Heirloom troff, to demonstrate us the typographic gain by `paragraph-at-once formatting´ against `line formatting´. And, as you seem to be the expert, please comment the visible differences. -- Thanks in advance. Joachim -- Dr. Joachim Walsdorff • Hauptstraße 225 • D 69117 Heidelberg • Tel. 06221-28680
Re: [Groff] Mission statement
On Tue 18 Mar 2014 12:58:09 Ingo Schwarze wrote: > Security-wise, PDF is > one of the most dangerous file formats, nowadays. That is true if the pdf reader you are using is configured to action all the extra bits which Adobe added to the standard (i.e. forms, flash and javascript). Without these "extras" it has the same risks as any other application consuming input from the web with regard to buffer overflows etc. I certainly would not use Adobe's Reader, slow and dangerous. Without these extras it is simply instructions to place marks on a canvas, much like svg, except that allows javascript. If you use gv to view postscript from the web you are actually running a postscript program in ghostscript so the attack surface is likely to be larger. Cheers Deri
Re: [Groff] Mission statement, second draft
Hello folks, Peter Schaffter wrote: > Here's the second draft of the mission statement, incorporating > suggestions from Ingo, Eric, Pierre-Jean, and others. It's starting > to come into focus, although a third pass will probably be necessary > before we commit to it. I mostly agree with that second mission statement. Nonetheless, I think that if the goal is to publish this mission statement in the hope that it encourages people to join the groff community, a bit more of « writing art » will be needed: words that encourage someone to come and work on groff. Werner spoke about the clean codebase of groff, we could also mention our experienced community and the precedence of Knuth and Kernighan algorythms. The mention of some "mentors" might also be important if some students whish to work on groff. Last but not least, the Gnu project might have some kind of help to give to groff developpers. That kind of literacy should probably be part of another document. > 2. Ingo's strategy... > 3. Eric's strategy... We all have a strategy here. I've got mine: find a path to minimize conflicts between heirloom troff and groff. These different strategies are part of our recent difficulties, and hopefully, we're gonna solve them. It's important that we do not discover in a few month that groff is a "cheval de Troie" for a goal that is finally not dedicated to the groff project. And I need some insurance that the man project is not such a "cheval de Troie". These insurances might be: - Don't forget the existing mdoc markup, - Study several solutions, - Don't use tricks to force the usage of that new markup but only arguments, - Show a draft of the whole project. This is a kind of literal hygiene mode for that discussion. Cheers, Pierre-Jean.
Re: [Groff] Mission statement, second draft
The section dealing with manpages had me hemming and hawing for
days. The original wording wasn't vague; it stated the matter
clearly--the intention to improve the semantic usefulness of
manpage markup. Details of strategy/implementation were omitted
because the issue is a minefield, and part of our mission will be
to navigate through it gracefully. Whether, as Eric proposes, we
tinker with man(7) so it plays nice with DocLifter, or, as Ingo
proposes, we advocate a migration to mdoc, the goal remains the
same: semantically clearer manpage markup for easier conversion to
xml. (Ingo spotted the reason for my general wording right off that
bat, as did a few others.)
Nevertheless, I've rewritten that section of the mission statement
outlining Eric's strategy for dealing with the manpage-markup=>xml
challenge. I don't expect it to be final, but here are my reasons.
(I'm non-partisan, merely trying to clear things up for a mission
statement.)
Hi Peter,
In the spirit of debate, I thought I'd dwell a bit more on the mdoc(7)
question. I think it's worth it!
My agenda is just to have good manpages. To me, good means portable
across systems and media, adhering to a common style, and having
human-readable source. Good on GNU systems, BSD, HTML, PS... "good".
We agree that man(7) doesn't cover that, I think. There's no standard
way of formatting non-trivial pages, resulting in a mix of roff(7) and
man(7) to compensate. The results are messy. What's worse is that the
community doesn't have a reliable, standard way of documenting works.
Everybody suffers. In other words, man(7) is *already* dead from the
perspective of good manpages. The proposal to extend it would be a new
macro package that shares some existing macros. But any non-trivial
manpage would effectively need to be re-written to take advantage of
these extensions.
Here are a few salient points in favour of mdoc(7), instead:
- mdoc(7) is the language for thousands upon thousands of manpages.
On my OpenBSD machine alone, the base system consists of >5000
semantically-searchable, HTML-renderable mdoc(7) manuals.
- mdoc(7) has been around for over twenty years, with continuous
attention to macros, behaviour, rendering, and so on. Twenty years of
real-world usage.
- It has considerable semantic power. Function names, variables,
standards compliance, and so on. I can search for variable types in all
manpages, or manpages adhering to a given standard. (What I'd give for
mdoc(7) versions of LAPACK's manuals...)
- It has mature tools for rendering and analysis, from the reference
implementation in groff(1) to full semantic search in mandocdb(8). (I
can't speak for other troffs... did Heirloom put out an mdoc package?)
- It has a thriving community of authors. Developers and users on
OpenBSD and NetBSD's misc@ lists use the languages on a daily basis.
Yes, mdoc(7) has cons. The macros themselves can be confusing. This
boils down to documentation: we've done a sucky job of documenting the
language itself. I've put a bit of effort into solving this with
http://manpages.bsd.lv, but there needs to be more to make it easy for
new manpage writers to get down to business.
And if some macros are hopeless, well that too, is a problem that can be
solved. First by careful documentation, then slowly, if absolutely
necessary, by way of deprecation. mdoc(7) isn't set in stone: its tools
are actively maintained and can be updated.
Peter, you'd mentioned that a new man(7) would give specific tasks to
developers. But you don't mention that mdoc(7) would require developers
to do nothing at all! The tools already exist and are mature. There's
no need for more work, unless to fit the underlying roff(7) in
groff(1)'s macro definition file to be more www-friendly. That would
need to happen anyway with a new man(7).
You also mention that Ingo's strategy, versus Eric's, involves social
engineering. How would this not apply to a new man(7)? Either way,
manpage authors will need to be educated, or re-educated, for the new
format.
I hear bits and pieces about how doclifter could help. Why can't
doclifter be taught to emit mdoc(7) instead of DocBook or this new
format? Wouldn't this solve everybody's problem? And if it can't, why
should those failings be relevant to a choice of manpage language?
Most significantly, the proposed format just doesn't exist. And there's
a lot of speculation on what it could be or wants to be with little tos
how for it. ("Semantics" has been thrown around a lot, but has been
left undefined.) Why not focus on mdoc(7), then consider a new man(7)
when it has had a few years in the field? A little competition is a
good thing, but at this point, you're stacking a known, stable product
against an idea. Especially since this idea could end up having the
same cons as mdoc(7)--maybe more. We just don't know.
1. No matter how successfully mdoc/mandoc have repla
Re: [Groff] Mission statement, second draft
On Tue, Mar 18, 2014, Joachim Walsdorff wrote: > it would be fine if you could provide an example text, formatted > both with groff and Heirloom troff, to demonstrate us the > typographic gain by `paragraph-at-once formatting´ against `line > formatting´. http://heirloom.sourceforge.net/doctools/just.pdf -- Peter Schaffter http://www.schaffter.ca
Re: [Groff] Mission statement, second draft
Peter Schaffter wrote: > On Tue, Mar 18, 2014, Joachim Walsdorff wrote: > > it would be fine if you could provide an example text, formatted > > both with groff and Heirloom troff, to demonstrate us the > > typographic gain by `paragraph-at-once formatting´ against `line > > formatting´. > > http://heirloom.sourceforge.net/doctools/just.pdf I've attached another postscript example produced by heirloom troff. Look at the blanks in the first paragraph, and how we can remove them so that the "color" of the paragraph seems more uniform. We can also use some tricks to increase or decrease the size of the whole paragraph and avoid orphans. Some art is still needed to adjust everything correctly. These features only offer the possibility to practice that art. Here is the code I used: .\" set heirloom extensions: .do xflag 3 .de lorem .sp 1v .\" text from wikipedia "microtypography". Microtypography is the name given to a range of methods for improving the readability and appearance of text, especially justified text. The methods reduce the appearance of large interword spaces and create edges to the text that appear more even. Heirloom Troff, the OpenType compatible free and open-source version of UNIX troff also supports protrusion, kerning and tracking. .. .vs 15 .ps 11 .ll 10v .po 3v .\" Historic troff behaviour .lorem .\" Paragraph at once adjustment .padj .lorem .sp |0 .po 16v .\" lightly use glyph size .letadj 99 99 15 101 101 .lorem .br .\" Define default interword space .ss 10 0 .\" Define minimal interword space .minss 9 .lorem Cheers, Pierre-Jean. f.ps Description: PostScript document
Re: [Groff] Mission statement, second draft
On Tue, 18 Mar 2014 10:29:50 -0400 Peter Schaffter wrote: > On Tue, Mar 18, 2014, Joachim Walsdorff wrote: > > it would be fine if you could provide an example text, formatted > > both with groff and Heirloom troff, to demonstrate us the > > typographic gain by `paragraph-at-once formatting´ against `line > > formatting´. > > http://heirloom.sourceforge.net/doctools/just.pdf > That is one interesting document...showing clearly the quality gain with paragraph-at-once formatting, as well as other things. The shaped paragraph facility is one that would greatly benefit groff. I did suggest it many years ago, but it wasn't taken up. I believe that it could be done with line formatting as well. Oh, and the PDF document above was beyond my version of Firefox's ability. I downloaded it: gv fails, finding errors and showing all the wrong glyphs; evince showed it fine but a page at a time. I regret to say that Adobe reader was the only one that correctly rendered as two-up. This also relates to the question of OTF and the need for a more sophisticated way in groff of referencing glyphs. For example I have a font with both regular caps and small caps, but the latter are called A_sc etc, so different encodings are required in the same font. Denis --
Re: [Groff] Mission statement, second draft
On Tue, Mar 18, 2014, Pierre-Jean wrote: > Nonetheless, I think that if the goal is to publish this > mission statement in the hope that it encourages people to > join the groff community, a bit more of « writing art » will > be needed: words that encourage someone to come and work on > groff. I agree with you about the need for something less dry. See further on. For the purposes of a mission statement, I tacked a note beside my monitor that said, "Avoid hucksterism." IOW, don't use the mission statement as selling job. > Werner spoke about the clean codebase of groff, we could > also mention our experienced community and the precedence of > Knuth and Kernighan algorythms. The mention of some > "mentors" might also be important if some students whish to > work on groff. Last but not least, the Gnu project might > have some kind of help to give to groff developpers. > > That kind of literacy should probably be part of another > document. Which document I will write, if others support the idea, and add to the groff page along with the mission statement. For all our discussions, our real mission right now is to get coders. We're all very good at our own bits and pieces of the groff picture, but none of us, I believe, is prepared to do the kind of C++ open-heart surgery we're proposing. As both Gunnar and Werner have reminded us, it's a BIG job. If I may muse here, briefly... I'm sick to death of groff always being the "also ran" to TeX. As every one of us knows, groff is terrific typesetting system, capable of tremendous flexibility and, with the right set of macros, wonderfully writer-friendly. Yet I'm fairly certain we've all encountered the "You did *that* with groff?!" reaction a time or two. Ignorance about groff as a complete typesetting system is practically pandemic. After five editions, O'Reilly's _Running Linux_ still demonstrates groff usage with a tutorial on writing manpages. And recently, I came upon this parenthetical comment at the Simon Fraser University site: "(I have a weirdly retrotech idea that we could do typesetting with groff. For regular prose, groff is every bit as powerful as TeX, while being about one tenth the size and complexity.)" If groff is as powerful as TeX while being one tenth the size, why on earth does the author dismiss it out-of-hand as weirdly retrotech? My feeling is that if groff can go head-to-head with TeX typographically, specifically wrt paragraph formatting, then we're in a much better position to combat the attitude implicit in comments like the one above and promote groff as a *contemporary* solution for typesetting needs. > And I need some insurance that the man project is not such a > "cheval de Troie". These insurances might be: > - Don't forget the existing mdoc markup, [a] > - Study several solutions, [b] [b] entails [a], methinks, and [b] is already taking place. > - Don't use tricks to force the usage of that new markup That's not going to happen. > - Show a draft of the whole project. Since Eric's taking leading on this, I imagine he'll do just that once debates are resolved and ideas have finished percolating. -- Peter Schaffter http://www.schaffter.ca
Re: [Groff] Add --with-doc configuration option
I wrote..
|find below something that adds the mentioned options and also
|tweaks the make system a bit -- 'make install' and 'make
|uninstall' seem to work again after the patch series is applied.
|There are still problems ('make install' with DESTDIR set results
|in some errors along the way, mostly contrib/hdtbl, 'make
|uninstall' still leaves some stale directories laying around)
[.]
For simplicity (i.e., passing by the GNU copyright scan :) i'll
attach the necessary add-on patch to make install and uninstall
completely error free, and without no stale directories laying
around.
The mentioned `contrib/hdtbl' problem is on the Apple Snow Leopard
side, however:
i am here 0: "/Volumes/UDF/arena/code.extern/groff.git/font/devps"
i am here 1: "all" *f is ""
i am here 1.5
i am here 2
i am here 3
i am here 4:, 22
i am here 4.5
examples/fonts_n.roff:153: Font not found.
^^
This because '.if !F \\*[*$1] \{\' -> '!F "\.."'
elapsed time: 0 seconds
***
...
examples/fonts_x.roff:153: name expected (got `\{')
listing font `'...
examples/fonts_x.roff:153: warning: can't find font `='
elapsed time: 0 seconds
***
That is, tr(1) is buggy, and the command
(from `hdtbl/examples/fonts_[nx].in')
. pso sh -c \
"echo -n .ds *f\ ; \
ls \\*[fontpath]/dev\*[.T] \
| tr '[:cntrl:]' ' '"
. \" This dummy line is necessary; the preceding line eats it.
doesn't work; if i do
0[steffen@sherwood groff.git]$ (cd font/devps/; ls| /usr/bin/tr '[:cntrl:]' '
')
then i get
r.pfa zapfdr.ps ?0[steffen@sherwood groff.git]$
however if i append `;echo' to the above it's fine. This doesn't
work in the code above, yet i decided not to look further since
this is Snow Leopard from i think 2009 and noone but me seems to
have complained about that? It's fine on the tested Linux and
BSD systems anyway.
--steffen
From 1345e758f55a5d883931eefdcfcb211778c23995 Mon Sep 17 00:00:00 2001
Message-Id: <1345e758f55a5d883931eefdcfcb211778c23995.1395161189.git.sdao...@users.sf.net>
From: "Steffen (Daode) Nurpmeso"
Date: Tue, 18 Mar 2014 15:43:12 +0100
Subject: [PATCH 1/2] src/roff/grog/Makefile.sub: fix typos missed in [3b934e4]
---
src/roff/grog/Makefile.sub | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/roff/grog/Makefile.sub b/src/roff/grog/Makefile.sub
index 91153af..b49d535 100644
--- a/src/roff/grog/Makefile.sub
+++ b/src/roff/grog/Makefile.sub
@@ -70,8 +70,8 @@ uninstall_sub:
-for f in $(GROG_); do \
$(RM) "$(DESTDIR)$(grog_dir)/$$f";\
done;
- -test -d "$(DESTDIR)$(grog_dir)" ||\
- -$(rmdir) "$(DESTDIR)$(grog_dir)"
+ -test -d "$(DESTDIR)$(grog_dir)" &&\
+ $(rmdir) "$(DESTDIR)$(grog_dir)"
# Emacs settings
--
1.9.0.dirty
From ef0bbed95b3290ec47600a579f42b7da39936ff1 Mon Sep 17 00:00:00 2001
Message-Id:
In-Reply-To: <1345e758f55a5d883931eefdcfcb211778c23995.1395161189.git.sdao...@users.sf.net>
References: <1345e758f55a5d883931eefdcfcb211778c23995.1395161189.git.sdao...@users.sf.net>
From: "Steffen (Daode) Nurpmeso"
Date: Tue, 18 Mar 2014 16:45:14 +0100
Subject: [PATCH 2/2] Tweak '$ make uninstall' to be free of errors
---
Makefile.comm | 23 ---
Makefile.in | 2 +-
contrib/chem/Makefile.sub | 4 +---
doc/Makefile.sub| 1 +
font/devpdf/Makefile.sub| 16
src/roff/groff/Makefile.sub | 5 +++--
src/roff/grog/Makefile.sub | 6 +++---
7 files changed, 29 insertions(+), 28 deletions(-)
diff --git a/Makefile.comm b/Makefile.comm
index d6f8df2..7de58bb 100644
--- a/Makefile.comm
+++ b/Makefile.comm
@@ -292,22 +292,23 @@ install_dev:
uninstall_dev:
-test -z "$(DEVFILES)$(DEVSCRIPTS)" ||\
for f in ""$(DEVFILES) $(DEVSCRIPTS); do \
- $(RM) $(DESTDIR)$(fontsubdir)/$$f;\
+ $(RM) $(DESTDIR)$(fontsubdir)/$${f};\
done
-test -z "$(OLDDEVFILES)" ||\
for f in ""$(OLDDEVFILES); do \
- $(RM) $(DESTDIR)$(oldfontsubdir)/$$f;\
+ $(RM) $(DESTDIR)$(oldfontsubdir)/$${f};\
done
- -test -d $(DESTDIR)$(fontsubdir)/generate &&\
- $(rmdir) $(DESTDIR)$(fontsubdir)/generate
- -test -d $(DESTDIR)$(fontsubdir)/enc &&\
- $(rmdir) $(DESTDIR)$(fontsubdir)/enc
- -test -d $(DESTDIR)$(fontsubdir)/map &&\
- $(rmdir) $(DESTDIR)$(fontsubdir)/map
- -test -d $(DESTDIR)$(fontsubdir)/util &&\
- $(rmdir) $(DESTDIR)$(fontsubdir)/util
+ -d="$(DESTDIR)$(fontsubdir)/generate";\
+ if test -d "$${d}"; then $(rmdir) "$${d}"; fi
+ -d="$(DESTDIR)$(fontsubdir)/enc";\
+ if test -d "$${d}"; then $(rmdir) "$${d}"; fi
+ -d="$(DESTDIR)$(fontsubdir)/map";\
+ if test -d "$${d}"; then $(rmdir) "$${d}"; fi
+ -d="$(DESTDIR)$(fontsubdir)/util";\
+ if test -d "$${d}"; then $(rmdir) "$${d}"; fi
-$(rmdir) $(DESTDIR)$(fontsubdir)
- -$(rmdir) $(DESTDIR)$(oldfontsubdir)
+ -d="$(DESTDIR)$(oldfontsubdir)";\
+ if test -d "$${d}"; then $(rmdir) "$${d}"; fi
.PHONY: depe
Re: [Groff] Add --with-doc configuration option
Hi Steffen, > That is, tr(1) is buggy tr(1) seems to be behaving as I'd expect. The \n is a control character and is being turned into space so you have to add another with echo. It thinks 0-0x1f and 0x7f are cntrl. $ recode /test8 tr -c '[:cntrl:]' - | > tr '[:cntrl:]' c | > fold -32; echo ---c $ > This doesn't work in the code above I can get an additional echo to work here, Linux, so I'm not quite sure what you're trying that fails? $ printf '%s\n' foo '.pso sh -c \' ' "echo -n a | tr a b"' bar | > nroff -U | grep . foo bbar $ $ printf '%s\n' foo '.pso sh -c \' ' "echo -n a | tr a b; \' ' echo"' bar | > nroff -U | grep . foo b bar $ Cheers, Ralph.
Re: [Groff] Add --with-doc configuration option
Heya, Ralph, Ralph Corderoy wrote: |Hi Steffen, | |> That is, tr(1) is buggy | |tr(1) seems to be behaving as I'd expect. The \n is a control character oh yes, compilation on CRUX-3 Linux succeeds without any problems, the font listing is worked from top to bottom. Now the only problem that remains for me is that $ ./configure --without-x --with-doc=examples; make fails with error if `gnu.xpm' cannot be converted because of missing tools. I'm still the opinion that this is not acceptable behaviour and that the `touch gnu.eps' should be automatic in such cases (after a warning message). |and is being turned into space so you have to add another with echo. It |thinks 0-0x1f and 0x7f are cntrl. | |$ recode /test8 tr -c '[:cntrl:]' - | |> tr '[:cntrl:]' c | |> fold -32; echo | | | |---c | | | | |$ | |> This doesn't work in the code above | |I can get an additional echo to work here, Linux, so I'm not quite sure |what you're trying that fails? | |$ printf '%s\n' foo '.pso sh -c \' ' "echo -n a | tr a b"' bar | |> nroff -U | grep . |foo bbar Hoppla. ?0[steffen@sherwood groff.git]$ printf '%s\n' foo '.pso sh -c \' ' \ "echo -n a | tr a b"' bar |nroff -U | grep . foo ‐n b bar | $ printf '%s\n' foo '.pso sh -c \' ' "echo -n a | tr a b; \' ' echo"' bar | |> nroff -U | grep . |foo b bar U-hu ?0[steffen@sherwood groff.git]$ printf '%s\n' foo '.pso sh -c \' ' \ "echo -n a | tr a b; \' ' echo"' bar |nroff -U | grep . foo ‐n b bar Ok but allow me to think a bit about those.. until at least tomorrow. :) Ciao |Cheers, Ralph. --steffen
Re: [Groff] Mission statement
Hi, Deri James wrote on Tue, Mar 18, 2014 at 01:26:02PM +: > On Tue 18 Mar 2014 12:58:09 Ingo Schwarze wrote: >> Security-wise, PDF is one of the most dangerous file formats, nowadays. > That is true if the pdf reader you are using is configured to action > all the extra bits which Adobe added to the standard (i.e. forms, > flash and javascript). I certainly don't trust my browser to keep all that disabled. > Without these "extras" it has the same risks as any other > application consuming input from the web with regard to buffer > overflows etc. Except the PDF format is more complex than most, offering much opportunity for bugs. Just saying, not everybody accepts PDF as a web document format, not only for lack of support in the tools, but also as a conscious choice. Yours, Ingo
Re: [Groff] Mission statement, second draft
Hi Peter, Peter Schaffter wrote on Mon, Mar 17, 2014 at 05:44:21PM -0400: > Here's the second draft of the mission statement, It is clearly maturing. [...] > The section dealing with manpages had me hemming and hawing for > days. The original wording wasn't vague; it stated the matter > clearly--the intention to improve the semantic usefulness of > manpage markup. Details of strategy/implementation were omitted > because the issue is a minefield, and part of our mission will be > to navigate through it gracefully. Whether, as Eric proposes, we > tinker with man(7) so it plays nice with DocLifter, or, as Ingo > proposes, we advocate a migration to mdoc, the goal remains the > same: semantically clearer manpage markup for easier conversion to > xml. Thanks for providing this summary; it helped me see more clearly. Actually, there are four questions that are somewhat separate but also influence each other a bit: (1) What are we to do with man(7)? Eric proposes to carefully evolve it to introduce a small amount of semantic markup. I propose to provide continuing support for backward compatibility and refrain from adding new features. (2) What are we to do with mdoc(7)? I propose to carefully maintain and polish it, of course without any substantial upheaval. If i understand Eric correctly, he is largely indifferent regarding mdoc(7). (3) What is our recommendation for manual writers, right now? I propose to tell them: If they want semantic markup right now, they can switch to mdoc(7), lightweight tools are ready for use. If i understand Eric correctly, he is telling them to continue writing their manuals with purely presentational man(7) markup for now and just rely on doclifter. (4) What is our vision for manual markup in, say, five years? Eric says, tell people to use new man-ext(7) features to be developed in the meantime. I say, just the same as today. If people don't care that much about semantic markup or don't mind heavy toolchains, they can leave their existing stuff in man(7) and use doclifter. If they do care, they can switch to mdoc(7) and use much simpler tools, just as today. Now, items (3) and (4) really need no decision in a groff mission statement. In item (2), i sense little potential for conflict. The fix is with item (1), really. What are the consequences of implementing Eric's plan regarding (1)? (1a) Whatever we do in groff_man(7), i have to re-implement it in mandoc(1). That will burn some of my time, and i don't relish the idea, but it's not a huge problem for me, either. mandoc(1) already supports a couple of man-ext(7) macros, .EX .EE .OP .UR .UE. Only .MT .ME .SY .YS .TQ are missing, simply because i never encountered them in any real-world manual so far. Usually, macros of Eric's design have been very easy to implement; well, .SY may be slightly tricky, but not that much worse than .TP, i guess. (1b) Whether or not we advertise new man(7) features, some authors will inevitably start using them, part of them *NOT* being aware that these are not portable. As long as the end-users run the software on platforms having groff(1) or mandoc(1), that's not a problem. However, users running on platforms having neither groff nor mandoc will suffer because these documents will misformat. That said, i'm not happy with a mission statement promoting man(7) feature additions, and i don't see the point, but i can grudgingly live with it. [...] > GROFF MISSION STATEMENT, 2014, 2nd draft > > As the most widely-deployed implementation of troff in use today, > GNU troff (groff) holds an important place in the Unix universe. > Along with TeX, it plays a leading role in the development of free > typesetting software for Unix systems. While maintaining backward > compatibility, it continues to evolve as a sophisticated typesetting > system with the advantages of small size, portability, and speed. > > Future groff development will focus on these areas: > > - improvements to the backend > - active development of general-purpose frontends > - the man(7) macros By your own argument (4) above, "I favour active development of both, letting evolutionary principles determine which is the more fit for an xml-based ecosystem", you might wish to say here: - the man(7) and mdoc(7) macros Or, if you prefer less technical wording: - ongoing development of manual page frontends or something similar. [...] > The man(7) macros Following your argument (4) above, you might wish to say something like Manual page macros instead, which would also make the subtitles more uniform. > The need for Unix manuals to render cleanly to multiple output media > favours the use of structural rather than presentational markup, but > the classical, widely-used man(7) macros are largely presentational. Th
Re: [Groff] Mission statement, second draft
Peter Schaffter : > Ignorance about groff as a complete typesetting system is > practically pandemic. After five editions, O'Reilly's _Running > Linux_ still demonstrates groff usage with a tutorial on writing > manpages. And recently, I came upon this parenthetical comment at > the Simon Fraser University site: > > "(I have a weirdly retrotech idea that we could do typesetting with >groff. For regular prose, groff is every bit as powerful as TeX, >while being about one tenth the size and complexity.)" > > If groff is as powerful as TeX while being one tenth the size, > why on earth does the author dismiss it out-of-hand as weirdly > retrotech? That's not a mystery to me. If it stays one to you, we have a problem; you can't plan to do anything effective against that perception if you don't understand it. Here are several reasons groff gets written off as "weirdly retrotech": * The [nt]roff markup design has a lot of tells that it was designed for very old machines that were ridiculously underpowered even by the standards of, oh, 1990, let alone today. Line-by-line formatting, short request names, limited number of font positions, no color support, a hole where embedded image support ought to be, things like that. Don't counter that groff fixed some of these things; that would be missing the point, which is that the core design screams "legacy!" * Sheer calendar age - a lot of people not sophisticated enough to spot those tells know it was written 40 years ago. * Strange, irregular, archaic-seeming markup design compared to XML or even TeX. Brian Kernignan called it "rebarbative" in *1979*. * Weak support for HTML output, no support for ePub. People on this list may think it's just fine that groff is so printer-oriented, but I promise you nobody else who was out of diapers by 1996 shares *that* quaint opinion. To put it more directly, groff seems like retrotech because it *is* retrotech. This creates a bit of a problem in trying to convince anyone otherwise. -- http://www.catb.org/~esr/";>Eric S. Raymond
Re: [Groff] Mission statement, second draft
Ingo Schwarze : > Actually, there are four questions that are somewhat separate > but also influence each other a bit: > > (1) What are we to do with man(7)? > Eric proposes to carefully evolve it to introduce a small amount > of semantic markup. > I propose to provide continuing support for backward compatibility > and refrain from adding new features. > > (2) What are we to do with mdoc(7)? > I propose to carefully maintain and polish it, of course without > any substantial upheaval. > If i understand Eric correctly, he is largely indifferent > regarding mdoc(7). > > (3) What is our recommendation for manual writers, right now? > I propose to tell them: If they want semantic markup right > now, they can switch to mdoc(7), lightweight tools are ready > for use. > If i understand Eric correctly, he is telling them to > continue writing their manuals with purely presentational > man(7) markup for now and just rely on doclifter. > > (4) What is our vision for manual markup in, say, five years? > Eric says, tell people to use new man-ext(7) features to be > developed in the meantime. > I say, just the same as today. If people don't care that much > about semantic markup or don't mind heavy toolchains, they can > leave their existing stuff in man(7) and use doclifter. If > they do care, they can switch to mdoc(7) and use much simpler > tools, just as today. I'm quoting this to confirm that it is a fair summary of my position. A few minor amendments: 1. I am at least as interested in narrowing and simplifying the man markup language (decoupling it from groff peculiarities) as I am in extending it. 2. What's in man_ext now may be enough extension already. I'm more inclined to think that than I was last week, 3. Wearing my "person who has to cope with interpreting it" hat, mdoc(7) annoys the crap out if me. But it is fair to say I am indifferent about its future relationship with groff. > What are the consequences of implementing Eric's plan regarding (1)? > > (1a) Whatever we do in groff_man(7), i have to re-implement it > in mandoc(1). That will burn some of my time, and i don't > relish the idea, but it's not a huge problem for me, either. > mandoc(1) already supports a couple of man-ext(7) macros, > .EX .EE .OP .UR .UE. Only .MT .ME .SY .YS .TQ are missing, > simply because i never encountered them in any real-world > manual so far. Usually, macros of Eric's design have been > very easy to implement; well, .SY may be slightly tricky, > but not that much worse than .TP, i guess. It is not an accident that they're easy to implement. I wanted them to be easy to understand. -- http://www.catb.org/~esr/";>Eric S. Raymond
Re: [Groff] Mission statement, second draft
On Tue, Mar 18, 2014, Kristaps Dzonsons wrote: > My agenda is just to have good manpages. To me, good means > portable across systems and media, adhering to a common style, and > having human-readable source. Good on GNU systems, BSD, HTML, > PS... "good". That puts us on the same page. :) > There's no standard way of formatting non-trivial pages, resulting > in a mix of roff(7) and man(7) to compensate. The results > are messy. What's worse is that the community doesn't have a > reliable, standard way of documenting works. Everybody suffers. > In other words, man(7) is *already* dead from the perspective of > good manpages. Agreed. > The proposal to extend it would be a new macro package that > shares some existing macros. Or "an existing macro package with additional macros." > But any non-trivial manpage would effectively need to be > re-written to take advantage of these extensions. As would converting man(7) formatted manpages to mdoc(7). As I see it, there are three strata of manpages: - legacy manpages =>those without a maintainer - current manpages =>those with an active maintainer - future manpages =>those yet to be written Legacy: Unless someone undertakes rewriting legacy manpages, they stay man(7) formatted forever and need DocLifter to output xml-ized versions. Think of DocLifter in this case as a kind of -Txml. (Eric was quite right when he accused us of "not being old-school enough.") Current: If maintainers re-write current manpages, they use whatever solution we come up with for "good manpages". If they don't re-write, we have a legacy condition. Future: Future manpages use mdoc(7) or the revamped man(7). If we subtract "current", we have a relatively straightforward situation (though not one free of debate): use DocLifter to get xml-ized versions of legacy manpages, and institute a policy shift toward a better markup system, whatever form that may take. In this scenario, your arguments in favour of mdoc would probably carry the day. However we do have current to deal with, and it's the crux of the debate. Which of "fix man(7)" or "switch to mdoc" puts the least burden of re-write on current manpages, and is therefore the most likely to receive community-wide acceptance? Let's imagine, for a moment, that man(7) is modified in such a way that backward compatibility with existing documents is fully supported, no presentational richness is lost, and extensions to the macro set permit clearer semantic structuring. With this imaginary man(7), authors who wished to make their manpages conformant wouldn't have much re-writing to do at all. Mostly it would involve replacing presentational clichés with semantic tags; in cases where there's a compelling need for presentational markup, groff would handle it as it always has. Hold-outs on rewrites wouldn't present much of a problem since their documents would render to -Tascii/-Tps/-Tpdf the same as always, and given the percentages Eric's provided about DocLifter, xml from the same documents would be >90% clean. Contrast that with converting manpages using man(7) to mdoc(7). I foresee considerable resistance and a very long slog ahead. While groff and mdoc are one place where a fruitful alliance exists between BSD and GNU, partisanship is factor that can't be ignored, however much I personally wish it could. > Peter, you'd mentioned that a new man(7) would give specific tasks > to developers. But you don't mention that mdoc(7) would require > developers to do nothing at all! Ah. I didn't express myself clearly. I consider it an *advantage* to have a project for developers to work together on right now. I truly believe groff is at a turning point, and honing our ability to function as a team is crucial. > You also mention that Ingo's strategy, versus Eric's, involves > social engineering. How would this not apply to a new man(7)? > Either way, manpage authors will need to be educated, or > re-educated, for the new format. I mentioned that Eric's strategy involved social engineering, too. My point was that he has a lot of experience with this. He knows the players and they know him. A substantial percentage have been responsive to his suggestions. His success rate is good. He can be very persuasive. :) These are x-factorish qualities we need in the quest for "good manpages." > I hear bits and pieces about how doclifter could help. Why can't > doclifter be taught to emit mdoc(7) instead of DocBook or this new > format? Wouldn't this solve everybody's problem? Not following you here. DocLifter takes man(7) and mdoc(7) as input and spits out DocBook xml. It wouldn't be emitting the "new format"; that would be its input. Or are you suggesting using DocLifter as a man(7)=>mdoc(7) converter? (I can see the blood draining from Eric's face.) > Most significantly, the proposed format just doesn't exist... > you're stacking a known, stable product against an idea. I'm aware. Just to be clea
Re: [Groff] Mission statement, second draft
On Tue, Mar 18, 2014, Eric S. Raymond wrote: > > Peter Schaffter : > > If groff is as powerful as TeX while being one tenth the size, > > why on earth does the author dismiss it out-of-hand as weirdly > > retrotech? > > That's not a mystery to me. If it stays one to you, we have a > problem; you can't plan to do anything effective against that > perception if you don't understand it. The question was more one of those lift your hands to heaven and cry "Why, god, oh why?" ejaculations than a real question. I know the reasons for groff's reputation. And I know a thing or two about combatting it. It's called mom. :) -- Peter Schaffter http://www.schaffter.ca
Re: [Groff] Mission statement, second draft
I am jumping in here, for which I apologize, because I have not had enough time over the last couple of months to become inovled in this discussion. All my spare time outside of my regular work during this period has been spent typesetting with groff: two newsletters, and a 200-plus-page book with over thirty graphs (using grap), and a program for a play, all using XML as markup and the best typesetting engine in the world (groff) for producing the output. > Peter Schaffter : > > > > "(I have a weirdly retrotech idea that we could do typesetting with > >groff. For regular prose, groff is every bit as powerful as TeX, > >while being about one tenth the size and complexity.)" > > > > If groff is as powerful as TeX while being one tenth the size, > > why on earth does the author dismiss it out-of-hand as weirdly > > retrotech? Peter is quoting John Maxwell, who has been trying for years now to get the publishing industry to think along the lines that Eric has been espousing (use structured markup). John and I have met several times to work out some methods and principles for typesetting XML documents, and he's very familiar about my arguments in favour of groff. His "weirdly retrotech" comment is tongue-in-cheek, and he's essentially making the same point as Peter. I agree with Peter that there's something very odd about the preference for TeX. I think it's likely just because that's what the vast majority of math and computer science students around the world have been told they must use in their documentation (theses, journal articles, etc.). About ten or so years ago, I produced about twenty books of computer science conference proceedings with TeX and LaTeX, and I think the system is overrated. The idea of the paragraph optimization is good, but the practice is a pain. When TeX fails in its efforts to justify a line, it fails spectacularly -- it oversets the line (i.e., into the margin). (Perhaps I'm ignorant about how to turn this "feature" off.) Trying to fix this is far more painful than in groff, because the kerning commands are very imprecise (e.g., "sloppypar"). The track kerning facility in groff allows adjustments, obviously, in 1/1000th of a point. I've always been able to fix problem paragraphs faster with groff. Anyone concerned with quality typesetting has to scan and make human judgements throughout the text. You can't rely on algorithms, although obviously they can reduce problems considerably. But even besides this, TeX is not a filter (so it does play well with other filters) and is very noisy. Groff is clean and agile compared to it. On Tue, Mar 18, 2014 at 06:13:11PM -0400, Eric S. Raymond wrote: > Subject: Re: [Groff] Mission statement, second draft > Here are several reasons groff gets written off as "weirdly retrotech": > > * The [nt]roff markup design has a lot of tells that it was designed > for very old machines that were ridiculously underpowered even by the > standards of, oh, 1990, let alone today. Line-by-line formatting, > short request names, limited number of font positions, no color > support, a hole where embedded image support ought to be, things like > that. Don't counter that groff fixed some of these things; that would > be missing the point, which is that the core design screams "legacy!" I don't think this is fair commentary. The command structure (request names, etc.) is irrelevant to the underlying algorithms, which were completely re-vamped by James Clarke circa 1989 (presumably on his Sun Workstations, hardly underpowered for 1990), with help from the intense work that SoftQuad did on the original AT&T code in the mid-eighties. As I've said above, my practical experience is that the line-by-line formatting does not make a big difference. As for "legacy", what about "ls", "cat", "grep"? > * Sheer calendar age - a lot of people not sophisticated enough to spot > those tells know it was written 40 years ago. Quality typesetting requires not just good tools but lots of experience. That hasn't changed since Gutenberg. I don't think the idea is to create a typesetting tool that looks fashionable. Adobe's got that market cornered. > * Strange, irregular, archaic-seeming markup design compared to XML or > even TeX. Brian Kernignan called it "rebarbative" in *1979*. Groff is a filter. The input language, the markup, etc., is entirely arbitary. I've been feeding groff SGML and XML since 1988 or 1989 and SoftQuad was doing it for sqtroff long before that. > * Weak support for HTML output, no support for ePub. People on this > list may think it's just fine that groff is so printer-oriented, but I > promise you nobody else who was out of diapers by 1996 shares *that* > quaint opinion. I have always agreed with your encouragement of creating documents in a structured markup language. There's nothing intrinsic to groff that prevents this. With a structured document you go straight to HTML or ePub through scripts; groff is irrelevant to this, ex
