Hi Graham and all Lilyponders,
I was quite astonished to hear that my slides were understood to mean
anything pessimistic or negative. If they give people this impression,
then it is a defect of the slides which I will fix.
But right now, let me address some of the apparent misunderstandings.
Are you seriously going to submit a paper to CMJ saying "a volunteer
open-source project has limited resources for fixing bugs" ???
This is not what we are saying in our presentation and/or paper.
What we are saying is: "We attempted a publication of a major critical
edition through a major publishing house, using software from a
volunteer open-source project with limited resources. We first hoped
that this software would be immediately (or almost immediately) suitable
for critical-edition work. We found issues blocking this work. These
issues are orthogonal to the main direction of the LilyPond project. We
fixed them, making the book possible. Future work includes making these
solutions useful for the wide LilyPond audience, not just for the
immediate needs of this particular book".
Oh, and I hope that this "paper to appear" includes an excellent
reason why you didn't use lilypond-book and LaTeX, which unlike
lilypond itself is*designed* to mix music and text.
Two reasons, really.
Reason one, we did investigate that route. Lilypond-book and LaTeX do
not achieve the kind of music/text integration that the editor
demanded. Another issue was integrating with their existing database of
variant readings. And for me, there was the obstacle that it was not
what the end user was asked for.
Reason two, the cry for help was heard all over the mailing list
starting many, many months ago. Anyone with a LaTeX-based solution
yet? Anyone even *suggesting* a LaTeX-based solution yet? After a year
and non-trivial (thousands of Euros) bounties offered?
I agree that the presentation has a disproportionately large first
(introductory) part, which makes it seem like a "presentation about
LilyPond", while it is really the second part which is the essence.
There is a background why I chose to extend this first part to be
somewhat detailed. Before the presentation at OCLUG, there were two
"micro-presentations" I did in a more informal setting. They ended in
somewhat curious Q&A discussions.
The first question -- asked by a guy who is well-known to everyone in
the digital typography and content engineering field through dozens of
papers and monographs -- he couldn't believe that today, in 2010, when
everything programmable into a computer had already been programmed,
today there are still books that can not be printed using already
written software. He demanded examples of problems (in music
typesetting) which still waited for their programming solutions. So 45
minutes were devoted to these examples.
At another micro-presentation, there was another question. "I thought
music typography was completely solved by Unicode? That is, all you
need to publish any score, is LaTeX and a font with all the music
symbols in it?" So, another 15 minutes to explain why LaTeX+Feta !=
music engraving system.
So this is the kind of questions typically asked by those in the
audience who care at all. This is why I felt an extended introduction
to LilyPond was needed -- although the real subject of the presentation
is mostly orthogonal to LilyPond per se.
his slides would discourage any potential
developers from joining
If there is a person who gets this impression from the slides, then the
slides are definitely defective and need fixing ASAP. I am not sure if
the presentation had the same defect or it's just the slides (there
certainly was a lot more explaining of the reasons and the background),
but they are most definitely not supposed to convey any negative ideas.
"The community showed no interest in addressing these issues", well I am
certainly not saying it's the community's *fault* -- the point is that
there are other dimensions, which are of concern to [at least one] top
publisher, in connection with [at least one] type of book, and which are
orthogonal to the dimension of the direction of work of the LilyPond
project. After all, none of these issues originated from me; they
originated from the musicologist doing the chant research, and the
editor of the book -- before I even started looking at LilyPond -- and
were rejected as not even being valid LilyPond problems. The typical
answers were,
"I think that you'll be better served by others"
http://www.mail-archive.com/lilypond-u...@gnu.org/msg52031.html
"Your post is absolutely unnecessary"
http://www.mail-archive.com/lilypond-u...@gnu.org/msg52334.html
"I recommend magic"
http://www.mail-archive.com/lilypond-u...@gnu.org/msg52026.html
Does this represent a problem enough to "discourage" people? I think it
does not, for two reasons. First, people are smart to take such things
with a grain of salt. If you look through the lkml archives through the
last decade, you will see a lot darker atmosphere in that community, yet
there are plenty of active kernel developers. Second, and I insist to
repeat it over and over, the main LilyPond project focus, and what we
are doing in our "Publisher-Grade" project, are different, orthogonal
dimensions.
"Patches were hard or impossible to contribute back" -- this is through
no fault of the community. In fact, working with Joe Neeman -- both to
initially produce patches, and to make them conform to the LilyPond
standards so they can be pushed into the main codebase, -- has been a
highest pleasure, and his help and guidance are priceless. His time and
patience are highly appreciated. The help from other developers, has
also been great (Neil, Carl, my sincere thanks!!) But this is not what
I am talking about; all this work together with Joe and others, it's all
about "the LilyPond dimension", while the problems blocking the
publisher, are in those other dimensions, and LilyPond being a volunteer
project, no one has incentives to do that not-very-core,
not-very-exciting work.
So it's a question of the proverbial "scratching one's own itch".
Simplest example: a patch fixes a bug (a Blocker for our real-life
project). The fix is used in production for some time, and seems to be
working fine. Code review on Rietveld, patch looks good to the
reviewers. The only problem delaying its push, is the absence of a test
case. Ok, I spend some time adding a test case, but bump into
problems. Joe is so kind to offer his help in debugging; but to get
through the debugging of the problem would seem to require half a day,
maybe a day. Do I understand the importance of having a test case?
Yes! Do I want a test case? Yes!!! But it's about customer value. "A
defect is anything that leads to customer dissatisfaction". Well, the
end-user does not care about having a test case. The time is very
tight, and we have other pressing priorities (like other Blocker bugs).
He would appreciate the inclusion of the work in the main LilyPond
codebase, but not to the extent of stopping work on the other Blocker
bug in order to write the test case so that the fix gets pushed to the
main repo. If it's an hour, yes. But five hours...let it sit in the
vendor tree.
There are also patches that require a lot more work to become
universally useful to all LilyPond users, not just to our book here.
When a patch takes 3 days from start to deployment in production, and
half a day to polish to be pushed into main repo, that's fine, I do that
half-day. When a patch takes the same 3 days, and would need another 5
days to be able to conform to LilyPond's standards, I might, but the
customer would not appreciate it, so *having* the patch would in effect
constitute a defect.
What's the way out of this?
Well I *hope* that by solving these "other-dimension" problems, we will
be able to make LilyPond a platform for serious projects which will
support its development. That way, we will be able to afford real
professional development; afford full-blown solutions, to meet all
standards, and be useful to the community at large; get everything from
my vendor tree into the main tree, and have no need for a vendor tree
any more. This is what I meant when I wrote that last "Future Work" slide.
On 06/12/2010 11:28 AM, Reinhold Kainhofer wrote:
Flowing text with music is simply out of the scope of lilypond. That would
mean that a full text layout system is implemented, too. Otherwise things like
hyphenation or widows/orphans cannot be properly handled. Ideally, of course,
we would have someone / some company working on that, but the resources are
simply limited.
Dear Dr. Kainhofer, your posts on the list have been very reassuring to
me, that my work on LilyPond is going in the right direction. What you
observe about your work on your critical editions, agrees very well with
my understanding of publisher-grade work.
Widows/orphans are already implemented (and even pushed! back in January
I think). Although the implementation is very limited (for example, the
length of the line is not taken into account), but the Editor is pleased
with the resulting look of the book, and then again, "a defect is
anything that leads to customer dissatisfaction".
Yes, the text layout system is extremely primitive. Even worse than
hyphenation, the line-breaker is a simply greedy word-wrapper, and worst
of all, the stretchability of the embedded music does not mingle well
with that of text. But again, the result pleases the Editor, and the
first volume of the book will be accepted with this slightly imperfect
stretching. If we get support to continue our work to improve LilyPond,
there will be further progress in the text layout system.
And, as you note, fragment line breaking is extremely complex...
The key here is to be very conscious of what limitations we are willing
to accept.
If we make our mind that we only have the resources to within the
limitations of the greedy line breaking, fragment line breaking suddenly
becomes very easy. We have it working in production since February.
I first tried lilypond, too,
but soon switched to latex. I don't have to mix music with text, but only
insert some figures
That's the thing -- we HAVE to mix music with text, and because if the
specifics of the plainchant we are dealing with, fragments HAVE to flow
seamlessly with the text.
I don't think there is zero interest in having the problems fixed, rather the
contrary. However, there is zero time to address them ourselves.
What I am trying to do, is create some sort of professional LilyPond
ecosystem, where people would be allowed to spend serious amount of time
on LilyPond work, but where problems would actually get fixed. If a
publishing project is willing to spend many thousand dollars to fix a
certain problem, and the only kind of response they get in a year, is "I
recommend magic" and "Your post is absolutely unnecessary", that kind of
interest-in-having-the-problems-fixed does not really matter to the
publishing project.
And, btw, I'm very interested in footnotes, as I'd need them with my critical
editions, too...
At this point, we temporarily put them on the back burner.
When the question about footnotes was asked on the list, no one raised
their hand saying "I'll do it". Plus, footnotes would be probably less
good in the book we are preparing than endnotes, because of the huge
amount of variants. If we place the critical apparatus in footnotes, it
will occupy almost the whole page, making the book difficult to use for
practical singing.
BTW, have you fixes for the vertical alignment been pushed yet?
You mean, the vertical space estimation? There were several bugs
screwing height-estimation. Some of those fixes are pushed, and some
are still sitting on Rietveld. Which ones specifically were you
interested in? Or do you mean the text / embedded score alignment? The
fix to that, requires as a prerequisite the "multi-line embedded score"
fix which is sitting on Rietveld.
Boris
_______________________________________________
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel
