Using/requiring Cairo

[David Kastrup]

Hi folks,

last time I looked at Cairo, its PDF generation was not really suitable
for use in LilyPond.  I have now updated my Cairo repository and saw
that there are commits for its PDF backend supporting hyperlinks,
document outlines, document metadata as of last October.

What does that mean?  Mainly a viable migration strategy where we might
be able to drop catering for a whole lot of graphics programming
ourselves by introducing a dependency on Cairo.  I am not overly
enthused about the programming quality of Cairo but LilyPond's quality
tends to be worse in the same departments and it's also hugely
inefficient due to using Scheme data structures and programming where
they are rather inappropriate and waste a humongous amount of resources.

It would be a large amount of work to bring LilyPond's graphics
programming up to scratch.  Moving to Cairo's data structures alone
would be quite advantageous and would likely speed up backend operations
significantly.

We might also be able to forego creating PostScript as an intermediate
stage to creating PDF and create bitmap formats without using PostScript
as well (again, this should really speed up things).

Drawbacks?  Like other things, being on Guile-2.x would make for
synergies with existing projects (not least of all guile-cairo though I
haven't checked its suitability in general, with the new PDF-level
features, and possible compatibility with Guile-1: I think it supported
Guile-1 at some point of time).

Of course, we have a number of other areas under construction anyway,
but I think that the graphics handling of LilyPond is so undercatered
for that our dearth of resources is more a reason for than against
starting on such a move.

The first step would likely just involve moving to Cairo data structures
while keeping most of the current code except where the code would
duplicate Cairo API calls in a reasonably straightforward way.

-- 
David Kastrup

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[Werner LEMBERG]

> The first step would likely just involve moving to Cairo data
> structures while keeping most of the current code except where the
> code would duplicate Cairo API calls in a reasonably straightforward
> way.

Sounds very sensible.  Looking around for other PDF generation
libraries, we don't have much other choices, it seems...


Werner

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[David Kastrup]

Werner LEMBERG  writes:

>> The first step would likely just involve moving to Cairo data
>> structures while keeping most of the current code except where the
>> code would duplicate Cairo API calls in a reasonably straightforward
>> way.
>
> Sounds very sensible.  Looking around for other PDF generation
> libraries, we don't have much other choices, it seems...

Cairo is not really a PDF generation library.  Its main advantage would
be that it buys us a reasonable representation of graphical objects and
their manipulation.  The main question mark would concern font handling
but I think it integrates with FreeType as well as Pango.

It would likely be harder to mess around with things like Woff font
support in SVG but the overall consistency and possibilities for better
hand-off and post-processing of the typesetting results would be quite
improved.

-- 
David Kastrup

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[Han-Wen Nienhuys]

On Sat, Jun 24, 2017 at 12:43 PM, David Kastrup  wrote:
> What does that mean?  Mainly a viable migration strategy where we might
> be able to drop catering for a whole lot of graphics programming
> ourselves by introducing a dependency on Cairo.  I am not overly

what "catering for graphic programming" mean? There is graphical
programming, but a lot of it is done already. Moving towards a new
library will be a big undertaking, so it could use more justification.

> enthused about the programming quality of Cairo but LilyPond's quality
> tends to be worse in the same departments

gee, thanks.

> and it's also hugely
> inefficient due to using Scheme data structures and programming where
> they are rather inappropriate and waste a humongous amount of resources.

Show some numbers? How much is "humongous", and how much faster will
it be with the snazzy new Cairo infrastructure?

$ time ./out/bin/lilypond input/regression/mozart-hrn-3.ly
GNU LilyPond 2.19.63
Processing `input/regression/mozart-hrn-3.ly'
Parsing...
Interpreting 
music...[8][16][24][32][40][48][56][64][72][80][88][96][104][112][120]
Preprocessing graphical objects...
Interpreting music...
Interpreting music...
Interpreting music...[8][16][24][32][40][48]
Preprocessing graphical objects...
Interpreting music...
Interpreting 
music...[8][16][24][32][40][48][56][64][72][80][88][96][104][112][120][128][136][144]
Preprocessing graphical objects...
MIDI output to `mozart-hrn-3.midi'...
MIDI output to `mozart-hrn-3-1.midi'...
MIDI output to `mozart-hrn-3-2.midi'...
Finding the ideal number of pages...
Fitting music on 3 or 4 pages...
Drawing systems...
Layout output to `/tmp/lilypond-vi9Khi'...
Converting to `mozart-hrn-3.pdf'...
GS ps->pdf done: took 0.4800 secs
Deleting `/tmp/lilypond-vi9Khi'...
PS done: took 0.6000 secs
Success: compilation successfully completed

real 0m3.225s
user 0m2.997s
sys 0m0.209s


Assuming you did everything optimally, you could shave off about 20%
of the end-to-end runtime in this case. A simpler alternative could be
to revisit how the PS is structured; it's hard to believe this is a
good way to place text, for example:

5.3676 -165.1853 moveto /TeXGyreSchola-Regular 3.44335938 output-scale
div selectfont
0.5804 0. 0. /exclam
0.7512 0. 0. /t
0.6146 0. 0. /i
0.5463 0. 0. /space

Also, the PS could be written to a pipe, so GS runs in parallel to
Lily. Some of the time in GS could overlap with LilyPond.

> It would be a large amount of work to bring LilyPond's graphics
> programming up to scratch.  Moving to Cairo's data structures alone
> would be quite advantageous and would likely speed up backend operations
> significantly.

which ones? In terms of graphics, the backend just does interval
unions and offsets (floating point comparisions and additions) through
the Stencil object. Unless you also restructure how objects are
grouped, it's going to be similar.

I can't recall seeing stencil evaluation ever figure prominently on a profile.

> We might also be able to forego creating PostScript as an intermediate
> stage to creating PDF and create bitmap formats without using PostScript
> as well (again, this should really speed up things).

that's the first realistic speed up; the PDF => PNG step seems expensive.

> Drawbacks?  Like other things, being on Guile-2.x would make for
> synergies with existing projects (not least of all guile-cairo though I
> haven't checked its suitability in general, with the new PDF-level
> features, and possible compatibility with Guile-1: I think it supported
> Guile-1 at some point of time).
>
> Of course, we have a number of other areas under construction anyway,
> but I think that the graphics handling of LilyPond is so undercatered
> for that our dearth of resources is more a reason for than against
> starting on such a move.
>
> The first step would likely just involve moving to Cairo data structures
> while keeping most of the current code except where the code would
> duplicate Cairo API calls in a reasonably straightforward way.

I would suggest trying make a GUILE binding of sorts for Cairo, adding
that in parallel to existing GS support, and hacking untili you have
feature parity. Then drop the GS support, and migrate the cairo data
structures more towards the core of the program as needed.

I strongly recommend being in a state where the Cairo support is
half-merged but only half-working.

timiing patch follows.
-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
diff --git a/scm/backend-library.scm b/scm/backend-library.scm
index 8eec320..932d80c 100644
--- a/scm/backend-library.scm
+++ b/scm/backend-library.scm
@@ -21,6 +21,7 @@
 
 (use-modules (scm ps-to-png)
  (scm paper-system)
+ (ice-9 format)
  (ice-9 optargs))
 
 (define-public (ly:system command)
@@ -73,7 +74,8 @@
 
 (define-public (postscript->pdf paper-width paper-height
 base-name tmp-name is-ep

Re: Using/requiring Cairo

[David Kastrup]

Han-Wen Nienhuys  writes:

> On Sat, Jun 24, 2017 at 12:43 PM, David Kastrup  wrote:
>> What does that mean?  Mainly a viable migration strategy where we might
>> be able to drop catering for a whole lot of graphics programming
>> ourselves by introducing a dependency on Cairo.  I am not overly
>
> what "catering for graphic programming" mean? There is graphical
> programming, but a lot of it is done already. Moving towards a new
> library will be a big undertaking, so it could use more justification.
>
>> enthused about the programming quality of Cairo but LilyPond's quality
>> tends to be worse in the same departments
>
> gee, thanks.

Shrug.  What do you expect of large amount of code that's written once
and basically left alone afterwards?  The SVG backend produces sort-of
experimental code (and still has font/spacing problems that "should not
be there").  There are significant remaining parts of the internals that
work with radians instead of degrees (like all graphics formats,
libraries and backends use, including PostScript/PDF), leading to
numerical pain because right angles have no numerically stable
representation.

Basically everything dealing with transformation matrices or
coefficients unpacks Scheme lists (instead of uniform arrays or records
matched to C data structures) into reals and back.

There is, both in C as well as in Scheme, code like

(define (make-bezier-sandwich-stencil coords thick)
   (make-path-stencil
   `(moveto
   ,(car (list-ref coords 0))
   ,(cdr (list-ref coords 0))
 curveto
   ,(car (list-ref coords 1))
   ,(cdr (list-ref coords 1))
   ,(car (list-ref coords 2))
   ,(cdr (list-ref coords 2))
   ,(car (list-ref coords 3))
   ,(cdr (list-ref coords 3))
 curveto
   ,(car (list-ref coords 4))
   ,(cdr (list-ref coords 4))
   ,(car (list-ref coords 5))
   ,(cdr (list-ref coords 5))
   ,(car (list-ref coords 0))
   ,(cdr (list-ref coords 0))
 closepath)
   thick
   1
   1
   #t))

that goes to a lot of effort to just juggle data from one Scheme
representation into another Scheme representation, without an actual
representation useful for efficient processing.

There is code like

(define (bezier-part-min-max x1 x2 x3 x4)
  ((lambda (x) (list (reduce min 1 x) (reduce max -1 x)))
   (map
(lambda (x)
  (+ (* x1 (expt (- 1 x) 3))
 (+ (* 3 (* x2 (* (expt (- 1 x) 2) x)))
(+ (* 3 (* x3 (* (- 1 x) (expt x 2
   (* x4 (expt x 3))
(if (< (+ (expt x2 2) (+ (expt x3 2) (* x1 x4)))
   (+ (* x1 x3) (+ (* x2 x4) (* x2 x3
(list 0.0 1.0)
(filter
 (lambda (x) (and (>= x 0) (<= x 1)))
 (append
  (list 0.0 1.0)
  (map (lambda (op)
 (if (not (eqv? 0.0
(exact->inexact (- (+ x1 (* 3 x3)) (+ x4 (* 3 
x2))
 ;; Zeros of the bezier curve
 (/ (+ (- x1 (* 2 x2))
   (op x3
   (sqrt (- (+ (expt x2 2)
   (+ (expt x3 2) (* x1 x4)))
(+ (* x1 x3)
   (+ (* x2 x4) (* x2 x3)))
(- (+ x1 (* 3 x3)) (+ x4 (* 3 x2
 ;; Apply L'hopital's rule to get the zeros if 0/0
 (* (op 0 1)
(/ (/ (- x4 x3) 2)
   (sqrt (- (+ (* x2 x2)
   (+ (* x3 x3) (* x1 x4)))
(+ (* x1 x3)
   (+ (* x2 x4) (* x2 x3)
   (list + -

which just is better done in C, and even better left in the
responsibility of people who specialize in maintaining a graphics
library rather than a music typesetting program.

Stencils and their compositions are represented in Scheme data
structures that take considerable time to traverse and interpret even
though they aren't actually traversed and interpret in Scheme itself all
that much.

We've had reports where large works exhausted a 32bit address space.

Cairo is just modeled around a different programming model and data
structures suitable for the processing that its API provides.

>> and it's also hugely inefficient due to using Scheme data structures
>> and programming where they are rather inappropriate and waste a
>> humongous amount of resources.
>
> Show some numbers? How much is "humongous", and how much faster will
> it be with the snazzy new Cairo infrastructure?
>
> $ time ./out/bin/lilypond input/regression/mozart-hrn-3.ly
> GNU LilyPond 2.19.63
> Processing `input/regression/mozart-hrn-3.ly'
> Parsing...
> Interpreting
> music...[8][16][24][32][40][48][56][64][72][80][88][96][104][112][120]
> Preprocessing graphical objects.

Re: Using/requiring Cairo

[karl]

David Kastrup:
...
> The main question mark would concern font handling
> but I think it integrates with FreeType as well as Pango.
...

 cairo has support for freetype
https://cairographics.org/manual/cairo-FreeType-Fonts.html

 pango has support for cairo:
https://developer.gnome.org/pango/stable/pango-Cairo-Rendering.html

///

Citating https://www.cairographics.org/manual/cairo-text.html

 Description

 The functions with text in their name form cairo's toy text API. The
 toy API takes UTF-8 encoded text and is limited in its functionality
 to rendering simple left-to-right text with no advanced
 features. That means for example that most complex scripts like
 Hebrew, Arabic, and Indic scripts are out of question. No kerning or
 correct positioning of diacritical marks either. The font selection
 is pretty limited too and doesn't handle the case that the selected
 font does not cover the characters in the text. This set of functions
 are really that, a toy text API, for testing and demonstration
 purposes. Any serious application should avoid them.

 The functions with glyphs in their name form cairo's low-level text
 API. The low-level API relies on the user to convert text to a set of
 glyph indexes and positions. This is a very hard problem and is best
 handled by external libraries, like the pangocairo that is part of
 the Pango text layout and rendering library. Pango is available from
 http://www.pango.org/.

Regards,
/Karl Hammar

---
Aspö Data
Lilla Aspö 148
S-742 94 Östhammar
Sweden
+46 173 140 57



___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[karl]

Han-Wen Nienhuys:
> On Sat, Jun 24, 2017 at 12:43 PM, David Kastrup  wrote:
> > What does that mean?  Mainly a viable migration strategy where we might
> > be able to drop catering for a whole lot of graphics programming
> > ourselves by introducing a dependency on Cairo.  I am not overly
> 
> what "catering for graphic programming" mean? There is graphical
> programming, but a lot of it is done already. Moving towards a new
> library will be a big undertaking, so it could use more justification.
...
> > It would be a large amount of work to bring LilyPond's graphics
> > programming up to scratch.  Moving to Cairo's data structures alone
> > would be quite advantageous and would likely speed up backend operations
> > significantly.
> 
> which ones? In terms of graphics, the backend just does interval
> unions and offsets (floating point comparisions and additions) through
> the Stencil object. Unless you also restructure how objects are
> grouped, it's going to be similar.

If no one else like to care for postscript, I can step in to handle it.

> > We might also be able to forego creating PostScript as an intermediate
> > stage to creating PDF and create bitmap formats without using PostScript
> > as well (again, this should really speed up things).

I use PS as the final format. Cairo can export to postscript but it's 
PS is not nice to read, lilypond PS is much better in that respect.

...
> > Drawbacks?  Like other things, being on Guile-2.x would make for
> > synergies with existing projects (not least of all guile-cairo though I
> > haven't checked its suitability in general, with the new PDF-level
> > features, and possible compatibility with Guile-1: I think it supported
> > Guile-1 at some point of time).

http://git.savannah.nongnu.org/cgit/guile-cairo.git/tree/README

Build dependencies
==

* Guile 1.8.0 or newer
  http://www.gnu.org/software/guile/
* Cairo 1.2.0 or newer
  http://cairographics.org/

...
> I would suggest trying make a GUILE binding of sorts for Cairo, adding
> that in parallel to existing GS support, and hacking untili you have
> feature parity. Then drop the GS support, and migrate the cairo data
> structures more towards the core of the program as needed.

There already are guile bindings for cairo:

 https://savannah.nongnu.org/projects/guile-cairo

Regards,
/Karl Hammar

---
Aspö Data
Lilla Aspö 148
S-742 94 Östhammar
Sweden
+46 173 140 57



___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[David Kastrup]

k...@aspodata.se writes:

> Han-Wen Nienhuys:
>> On Sat, Jun 24, 2017 at 12:43 PM, David Kastrup  wrote:
>> > What does that mean?  Mainly a viable migration strategy where we might
>> > be able to drop catering for a whole lot of graphics programming
>> > ourselves by introducing a dependency on Cairo.  I am not overly
>> 
>> what "catering for graphic programming" mean? There is graphical
>> programming, but a lot of it is done already. Moving towards a new
>> library will be a big undertaking, so it could use more justification.
> ...
>> > It would be a large amount of work to bring LilyPond's graphics
>> > programming up to scratch.  Moving to Cairo's data structures alone
>> > would be quite advantageous and would likely speed up backend operations
>> > significantly.
>> 
>> which ones? In terms of graphics, the backend just does interval
>> unions and offsets (floating point comparisions and additions) through
>> the Stencil object. Unless you also restructure how objects are
>> grouped, it's going to be similar.
>
> If no one else like to care for postscript, I can step in to handle it.

I don't know what that means.  My proposed migration plan would not have
changed the PostScript backend at first, but it certainly would have
been slated for eventual retirement.

>> > We might also be able to forego creating PostScript as an
>> > intermediate stage to creating PDF and create bitmap formats
>> > without using PostScript as well (again, this should really speed
>> > up things).
>
> I use PS as the final format.  Cairo can export to postscript but it's
> PS is not nice to read, lilypond PS is much better in that respect.

PostScript is not intended to be human-readable rather than
human-writable and streamable, two characteristics PDF no longer cares
about in return for better computer-readability and processability.

LilyPond PostScript defines a whole lot of its own macros and passes
stuff for PDF processing.  That makes it a lot less likely to be
suitable for mechanical processing (like using "Tailor") than PostScript
generated from a general-purpose representation with commonly used
toolkits.

>> > Drawbacks?  Like other things, being on Guile-2.x would make for
>> > synergies with existing projects (not least of all guile-cairo
>> > though I haven't checked its suitability in general, with the new
>> > PDF-level features, and possible compatibility with Guile-1: I
>> > think it supported Guile-1 at some point of time).
>
> http://git.savannah.nongnu.org/cgit/guile-cairo.git/tree/README
>
> Build dependencies
> ==
>
> * Guile 1.8.0 or newer
>   http://www.gnu.org/software/guile/
> * Cairo 1.2.0 or newer
>   http://cairographics.org/

Well, it's maintained by Andy Wingo.  That makes it more likely to
update to newer Guile versions than newer Cairo versions.  But at least
configure.ac appears to support Guile 1.8 still explicitly.

> ...
>> I would suggest trying make a GUILE binding of sorts for Cairo, adding
>> that in parallel to existing GS support, and hacking untili you have
>> feature parity. Then drop the GS support, and migrate the cairo data
>> structures more towards the core of the program as needed.
>
> There already are guile bindings for cairo:
>
>  https://savannah.nongnu.org/projects/guile-cairo

One would have to evaluate whether their data structures work well for
the intent of providing a good data representation for use internal to
LilyPond's graphics processing.  If we end up with a whole lot of
back-and-forth conversion (and particularly lots of small storage
allocations) after all for crossing Scheme/C/C++ boundaries, it might
still make more sense to create one's own wrapping layer.  If the wrap
is a good fit, it might be good to make use of.

-- 
David Kastrup

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[karl]

David Kastrup:
> k...@aspodata.se writes:
> 
> > Han-Wen Nienhuys:
> >> On Sat, Jun 24, 2017 at 12:43 PM, David Kastrup  wrote:
...
> > If no one else like to care for postscript, I can step in to handle it.
> 
> I don't know what that means.

It's like english, I am willing to take care of something related to
the programming language PostScript, so we don't need to convert to
cairo.

>  My proposed migration plan would not have
> changed the PostScript backend at first, but it certainly would have
> been slated for eventual retirement.

And I more or less said that I didn't like the retirement thing.
If you already decided to switch to cairo, then fine, tell us so 
we can stop this discussion.

> >> > We might also be able to forego creating PostScript as an
> >> > intermediate stage to creating PDF and create bitmap formats
> >> > without using PostScript as well (again, this should really speed
> >> > up things).
> >
> > I use PS as the final format.  Cairo can export to postscript but it's
> > PS is not nice to read, lilypond PS is much better in that respect.
> 
> PostScript is not intended to be human-readable rather than
> human-writable and streamable,

PostScript is a programming language suited for printing and other 
things. Why youldn't I like to have that readable and understandable 
like any other code ?

> two characteristics PDF no longer cares
> about in return for better computer-readability and processability.

PDFReference16.pdf is 1236 pages long and is no easy read.

The first version of the green/blue/red book was good at getting you 
started at postscript, do you have any similar doc to get you started
at pdf ?

Yes, they say that pdf it is for better computer-readability and
processability, but it isn't a stable interface, things are being
added which breaks some viewers, and fonts are still missing in
some cases, which was a point going from ps to pdf. In contrast to
ps which has a stable interface/reference, PLRM3 is first printed
1999 and still in effect.

I have tried to find some free lib to use to read pdf's, if you know
of one please tell me, preferable for perl. Currently I'm using
"pdftohtml --xml", but there is a few things missing with that solution.

I know about poppler, citing:
  https://freedesktop.org/wiki/Software/poppler/
//
 Documentation

 There is currently very little documentation. 
//

Soo, the computer-readability is a nice goal, but not so much attained.

...
>  That makes it a lot less likely to be
> suitable for mechanical processing (like using "Tailor") than PostScript
> generated from a general-purpose representation with commonly used
> toolkits.
...

Ok, stop bullshitting. Both ps and pdf are a "general-purpose 
representaion", both have "commonly used toolkits", and both are 
suitable for "mechanical processing" (by e.g. a printer).

What is "Tailor" in this context, I don't understand what you mean by 
that ?

Regards,
/Karl Hammar

---
Aspö Data
Lilla Aspö 148
S-742 94 Östhammar
Sweden
+46 173 140 57



___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[David Kastrup]

k...@aspodata.se writes:

> David Kastrup:
>> k...@aspodata.se writes:
>> 
>> > Han-Wen Nienhuys:
>> >> On Sat, Jun 24, 2017 at 12:43 PM, David Kastrup  wrote:
> ...
>> > If no one else like to care for postscript, I can step in to handle it.
>> 
>> I don't know what that means.
>
> It's like english, I am willing to take care of something related to
> the programming language PostScript, so we don't need to convert to
> cairo.

"taking care of PostScript" is not related to converting LilyPond's
graphics internals to Cairo since LilyPond's graphics internals are not
written in PostScript.

>>  My proposed migration plan would not have
>> changed the PostScript backend at first, but it certainly would have
>> been slated for eventual retirement.
>
> And I more or less said that I didn't like the retirement thing.

The TeX backend processing was retired when LilyPond was changed from
version 1 to 2 and learnt to write its own PostScript.

At the current point of time, high amounts of work are invested into the
font processing in PostScript, dealing with subletting, with document
merging and so on.  When processing in general will get moved to Cairo,
there will eventually be a state where

a) the backends are organized differently
b) ongoing work is generally invested only in the Cairo backend path

That means that anybody willing to "care for PostScript" in the sense of
producing the same kind of PostScript output as now will have a whole
lot more work on his hand than just basic maintenance of existing code
since he has to replace all the experts currently invested with the
finer points of PostScript intended for PDF conversion while they move
on.

> If you already decided to switch to cairo, then fine, tell us so we
> can stop this discussion.

Shrug.  I don't have any capacity to decide anything.

>> > I use PS as the final format.  Cairo can export to postscript but it's
>> > PS is not nice to read, lilypond PS is much better in that respect.
>> 
>> PostScript is not intended to be human-readable rather than
>> human-writable and streamable,
>
> PostScript is a programming language suited for printing and other
> things. Why youldn't I like to have that readable and understandable
> like any other code ?

Because it is employed as an intermediate format not intended for human
consumption and manipulation?  It's similar to the role of LilyPond in a
Denemo workflow.  It's nice for debugging when it's somewhat readable,
but the files created in it are ephemeral.  Editing them does not make
sense.  PDF has by far outclassed PostScript in usage _because_ it does
no longer bother being readable and writable by humans rather than
programs.

>> two characteristics PDF no longer cares about in return for better
>> computer-readability and processability.
>
> PDFReference16.pdf is 1236 pages long and is no easy read.
>
> The first version of the green/blue/red book was good at getting you
> started at postscript, do you have any similar doc to get you started
> at pdf ?

You are not supposed to get started at writing PDF.  Instead the PDF
gets written by programs.  You are fine with letting your PDF be written
by Ghostscript rather than LilyPond and not looking at it.  What is the
difference with PostScript in your workflow that makes it different?

> I have tried to find some free lib to use to read pdf's, if you know
> of one please tell me, preferable for perl.

libpoppler?

> Currently I'm using "pdftohtml --xml", but there is a few things
> missing with that solution.
>
> I know about poppler, citing:
>   https://freedesktop.org/wiki/Software/poppler/
> //
>  Documentation
>
>  There is currently very little documentation. 
> //
>
> Soo, the computer-readability is a nice goal, but not so much
> attained.

Well, there isn't a free lib for reading PostScript, is there?

I mean, how much is ps2pdf documented?

> ...
>>  That makes it a lot less likely to be
>> suitable for mechanical processing (like using "Tailor") than PostScript
>> generated from a general-purpose representation with commonly used
>> toolkits.
> ...
>
> Ok, stop bullshitting. Both ps and pdf are a "general-purpose 
> representaion", both have "commonly used toolkits", and both are 
> suitable for "mechanical processing" (by e.g. a printer).
>
> What is "Tailor" in this context, I don't understand what you mean by 
> that ?



Basically the principal reason NeXT computers were kept around (NeXT was
what Steven Jobs was working on while being laid off from Apple).

PDF editing was more dependable.

-- 
David Kastrup

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[karl]

David Kastrup:
> k...@aspodata.se writes:
> > David Kastrup:
...
> "taking care of PostScript" is not related to converting LilyPond's
> graphics internals to Cairo since LilyPond's graphics internals are not
> written in PostScript.

Ok, forget it then, you are not listening.

...
> When processing in general will get moved to Cairo,
...

You make it sound like it is already decided.

...
> > PostScript is a programming language suited for printing and other
> > things. Why youldn't I like to have that readable and understandable
> > like any other code ?
> 
> Because it is employed as an intermediate format not intended for human
> consumption and manipulation?  It's similar to the role of LilyPond in a
> Denemo workflow.  It's nice for debugging when it's somewhat readable,
> but the files created in it are ephemeral.
...

Fine, I also want them to be easy to debug.
PS is not ephemeral for me.

...
> You are not supposed to get started at writing PDF.  Instead the PDF
> gets written by programs.  You are fine with letting your PDF be written
> by Ghostscript rather than LilyPond and not looking at it.  What is the
> difference with PostScript in your workflow that makes it different?

You don't listen, I use the PS as is, not doing any conversion to pdf
unless someone else wants them.

And my interest was in reading pdf's, so I can write programs to extract
the info I want from pdf's. I took up the subject to show that the goal
of computer-readability and processability is not reached.

> > I have tried to find some free lib to use to read pdf's, if you know
> > of one please tell me, preferable for perl.
> 
> libpoppler?

Yes, but it is barely documented, makes it hard to use.

> > Currently I'm using "pdftohtml --xml", but there is a few things
> > missing with that solution.
> >
> > I know about poppler, citing:
> >   https://freedesktop.org/wiki/Software/poppler/
> > //
> >  Documentation
> >
> >  There is currently very little documentation. 
> > //
> >
> > Soo, the computer-readability is a nice goal, but not so much
> > attained.
> 
> Well, there isn't a free lib for reading PostScript, is there?

I do have an editor.

> I mean, how much is ps2pdf documented?
...

Ok, you don't know then, so much for computer-readability and
processability.

Regards,
/Karl Hammar

---
Aspö Data
Lilla Aspö 148
S-742 94 Östhammar
Sweden
+46 173 140 57



___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[Werner LEMBERG]

>> "taking care of PostScript" is not related to converting LilyPond's
>> graphics internals to Cairo since LilyPond's graphics internals are
>> not written in PostScript.
> 
> Ok, forget it then, you are not listening.  [...]

Why such a hostile tone, Karl?  There is no reason for this.

I have to agree with David: While having well readable PS is nice,
this is just a by-product of the conversion process from a .ly to a
.pdf file.  Essentially, the formatting of the PS stuff is of *no
importance* to a vast majority of users.

> I use the PS as is, [...]

So tell us what you are doing with the PS stuff.

> And my interest was in reading pdf's, so I can write programs to
> extract the info I want from pdf's.

Have you tried `pdftk' to uncompress a PDF file?  It's then almost as
readable as a plain PS file (see attachments).


Werner



ly-crop.pdf
Description: Adobe PDF document


ly-crop.uncompressed.pdf
Description: Adobe PDF document
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[karl]

Werner:
> >> "taking care of PostScript" is not related to converting LilyPond's
> >> graphics internals to Cairo since LilyPond's graphics internals are
> >> not written in PostScript.
> > 
> > Ok, forget it then, you are not listening.  [...]
> 
> Why such a hostile tone, Karl?  There is no reason for this.

No, that is not a hostile tone, it's a tired tone.

> I have to agree with David: While having well readable PS is nice,
> this is just a by-product of the conversion process from a .ly to a
> .pdf file.  Essentially, the formatting of the PS stuff is of *no
> importance* to a vast majority of users.

I don't argue about that, save if you just want the ps file.

> > I use the PS as is, [...]
> 
> So tell us what you are doing with the PS stuff.

Mostly the same thing you do with your pdf's, I guess, like
psnup, psselect, pstops, a patched version of psbook (so the paper 
curves the right way when I fold them out of the printer),
includegraphics from latex, and probably some more. And my printers
doesn't take pdf.

> > And my interest was in reading pdf's, so I can write programs to
> > extract the info I want from pdf's.
> 
> Have you tried `pdftk' to uncompress a PDF file?  It's then almost as
> readable as a plain PS file (see attachments).

No, it looks promising, thanks, now I have to find out how it does
that.

Though, my conclusion was that pdf is not better than ps regarding
postprocessing, and yes I know that pdf (depending on pdf version)
has some features that ps don't have.

Regards,
/Karl Hammar

---
Aspö Data
Lilla Aspö 148
S-742 94 Östhammar
Sweden
+46 173 140 57



___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: I'm to teach LilyPond officially again

[Trevor]

As the maintainer of LilyBin, I'll be interested in knowing if you
encounter any issues, especially with reliability of the service! Last
year, I and another developer did some work to get LilyBin running on AWS
Lambda especially to handle the load of classroom situations where many
people use are compiling concurrently. Let me know how it goes!

On Sun, Jun 18, 2017 at 6:05 PM Francisco Vila 
wrote:

> Hello all,
>
> Some time ago I was teaching LilyPond in a Conservatoire, then stopped
> doing so, now I'm doing it again! The course is October to June and it's
> optional and open to all four levels of High Music Education.
>
> I'm delighted because I was already missing giving lectures to groups. I
> plan to keep all as practice-oriented as possible with a minimum of
> theoretical issues beyond what's strictly necessary to understand some
> issues.
>
> I have plenty of teaching material, including my 30 weekly lessons plus
> lesson zero.
>
> "Lesson Zero" is {}{}~~~  in a Spanish keyboard, I think
> once achieved that, all else is easy :-)
>
> I'll use Frescobaldi, lilybin, OLy for OOo, and more. Students will also
> learn to enter LP music in wikipedia.
>
> Any suggestions are welcome.
>
> Regards,
>
> --
> Francisco Vila. Badajoz (Spain)
> paconet.org , csmbadajoz.com
>
>
> ___
> lilypond-devel mailing list
> lilypond-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/lilypond-devel
>
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Mac OS X executable

[Andrew Bernard]

Is there a 64 bit OS X version of the lilypond application? If not, is
there any reason for that?

Andrew
___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

pdfsizeopt can now handle lilypond PDFs

[Werner LEMBERG]

Folks,


the author of pdfsizeopt

  https://github.com/pts/pdfsizeopt

has recently applied some bug fixes to make it work with lilypond
documentation files (cf. https://github.com/pts/pdfsizeopt/issues/11).

Using trueroad's approach (i.e., using `extractpdfmark'),
`lilypond-notation.pdf' generated from git on Feb. 27th has a size of
10019KByte.  After running pdfsizeopt, the size is reduced to
7962kByte; making the file smaller by 20%.

[As a side note, `notation.pdf' had a size of 24MByte six years
ago...]

I think it's a good idea to apply pdfsizeopt to all PDF files uploaded
to lilypond's web page.


Werner

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel

Re: Using/requiring Cairo

[David Kastrup]

k...@aspodata.se writes:

> Werner:
>> >> "taking care of PostScript" is not related to converting LilyPond's
>> >> graphics internals to Cairo since LilyPond's graphics internals are
>> >> not written in PostScript.
>> > 
>> > Ok, forget it then, you are not listening.  [...]
>> 
>> Why such a hostile tone, Karl?  There is no reason for this.
>
> No, that is not a hostile tone, it's a tired tone.
>
>> I have to agree with David: While having well readable PS is nice,
>> this is just a by-product of the conversion process from a .ly to a
>> .pdf file.  Essentially, the formatting of the PS stuff is of *no
>> importance* to a vast majority of users.
>
> I don't argue about that, save if you just want the ps file.
>
>> > I use the PS as is, [...]
>> 
>> So tell us what you are doing with the PS stuff.
>
> Mostly the same thing you do with your pdf's, I guess, like
> psnup, psselect, pstops, a patched version of psbook (so the paper 
> curves the right way when I fold them out of the printer),
> includegraphics from latex, and probably some more.

All of those work with PostScript created from Cairo.  All of those
utilities are also available for PDF.  Indeed, since they do not have to
rely on DSC, they work more reliably.

> And my printers doesn't take pdf.

They do take PostScript created by Cairo.  And likely print it faster
than the current LilyPond PostScript output since they don't have to
expand a number of complex commands themselves.

>> > And my interest was in reading pdf's, so I can write programs to
>> > extract the info I want from pdf's.
>> 
>> Have you tried `pdftk' to uncompress a PDF file?  It's then almost as
>> readable as a plain PS file (see attachments).
>
> No, it looks promising, thanks, now I have to find out how it does
> that.

Depending on what "Extract the info you want from PDFs" means, pdf2dsc
can be quite handy.  Basically, it gives you DSC-compliant file only
containing references into the PDF file.  That does not actually
convert/rewrite any content but yields to most PostScript manipulation
with results that are useful to processing with Ghostscript.

However, so should Cairo-generated PostScript do: using the PostScript
backend of Cairo does not run through PDF in the first place.

> Though, my conclusion was that pdf is not better than ps regarding
> postprocessing, and yes I know that pdf (depending on pdf version) has
> some features that ps don't have.

But nobody is talking about PDF.  The difference in question is
Cairo-generated PostScript vs the PostScript generated by Scheme
programming and LilyPond's music-drawing-routines.ps and encodingdefs.ps
.

-- 
David Kastrup

___
lilypond-devel mailing list
lilypond-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-devel