Hi Tim, I'm glad you're continuing the conversation, which has helped me at least to clarify my thinking about not just LP but about the nature of programming and the sort of tools we (or I at least) need to support good programming. I end up in a very different place than you, but I don't think it's a zero-sum game. Please take the comments below in the irenic spirit in which they are offered - not "you are wrong on so many levels that I'm compelled to enlighten you", but "here are some other ways to look at things that seem interesting."
On Wed, May 21, 2014 at 6:03 AM, <d...@axiom-developer.org> wrote: > The primary focus of a "documentation system" is communication from > the author to the audience. > Communication is an important part of the puzzle, I agree; but what exactly is it? There are lots of models<http://en.wikipedia.org/wiki/Models_of_communication>, which carry very different implications. > > One of the struggles apparent in discussing issues of communication, > especially with programmers, is Heideggers "present-at-hand" vs > "breaking down". > > Programmers write programs to instruct a machine to perform some > task. In the ideal, this involves "communication" from the mind of > the programmer to the "execution of the program". This suggests a transference or transmission model of communication based on what Roy Harris <http://www.royharrisonline.com/INP26.html> calls the "telementation" and fixed-code fallacies - the idea that communication involves transference of some kind of content ("thoughts", "concepts", etc.) from one mind to another, or to a machine, by encoding, sending, receiving, and decoding "messages". That may be a good model for computational communication, but it has severe (and I think insurmountable) problems as a model for human communication. More fundamentally, whether human-computer interaction should be considered communication at all is problematic to say the least. For a terrific study on communication check out Michael Tomasello's Origins of Human Communication<http://books.google.com/books/about/Origins_of_Human_Communication.html?id=T3bqzIe3mAEC>. The suggestion there is that successful communication amounts to successful coordination of joint action. The notion that programs "instruct" a machine to perform some task is very common - it's arguably central to Knuth's concept of LP - but I'm not convinced it is correct. In a sense, a program just *is* a machine, albeit an abstract one. When you compile it, you just translate it into another language, yielding an equivalent machine, one that a physical machine "understands". When you "run" the machine program, it doesn't "instruct" the hardware to do this or that, any more than the laws of physics "instruct" apples to fall in a certain way, or a hurled stone "instructs" the window it meets to shatter. ...snip... The effect of this constant "breaking down" is that we have learned, > rather painfully, to be aware of the machinery of the process at every > step of the way. This focus on the machinery becomes the expected way > of communicating with the machine. Scratch any programmer, interview at > any company, listen to any talk, and you find "machinery". > How could it be otherwise? Programming is machine construction. > > But communication from the author to the audience is the underlying > theme of literate programming. Knuth's point is about communication, > not about the machinery of communication. The question is, to what > audience, not how. > I'm not sure what "machinery of communication" means. It's not something I ever think about when I'm programming; in fact I never think about communication. I think about language, logic, and expressiveness. Whether others understand what I write is not something I can control; all I can do is try to write clearly under the constraints of the language I'm using. The problem with focusing on communication is that successful communication is the responsibility of all parties involved. Authors can be held responsible for what they write, but not for what readers read; this is true for all forms of writing. There's not much an author can do to prevent readers from misreading, other than try to write clearly. Unfortunately, the Principle of Charity<http://en.wikipedia.org/wiki/Principle_of_charity>is routinely ignored (especially on the internet!) Knuth: "Instead of imagining that our main task is to instruct a *computer*what to do, let us concentrate rather on explaining to *human beings* what we want a computer to do." Notice how Knuth changes horses in midstream, from "instructing a computer" (i.e. writing code) to "explaining to humans", which is not writing code. The conclusion that many (well, some) have drawn is that we should treat the "literary explanation" as central, and the code as secondary or even incidental. I guess that's ok if your task is to explain a program, but as a model of what programming is or ought to be, it's not even wrong. The *only* reason to write a (real, not toy or pedagogical) program is so that a (physical) machine can run it. If it can't be run, or if it produces the wrong result, then it doesn't matter how good the documentation is. Here's the Knuth quote from his LP article nobody ever cites: ""In fact, my enthusiasm is so great that I must warn the reader to discount much of what I shall say as the ravings of a fanatic who thinks he has just seen a great light." We should take him seriously. [Also, as a historical matter, Knuth himself clearly had strong ideas about the "how" involving things like "natural" narrative order, monolithic text+code, etc.; see the quotes at https://github.com/mobileink/codegenres/wiki/Quotations.] The notion that programmers should focus on documentation addressed to humans is where I part company with Knuth (and just about everybody else, I reckon). Time spent writing "magnificent" documentation is time that could be spent making the code better and more understandable. I actually think that Knuth completely misunderstood the nature of programming. I do *not* think that programming **essentially** involves "communication" with other humans (although it arguably does involve coordination with other humans). Or with machines, for that matter. The essence of programming is the construction of machines, which is something that creatures lacking natural language can do in principle and do in fact do - there are many examples of non-human animals constructing and/or using machines (e.g. primates using sticks to pull termites out of a mound). When you compile and run a program there is no communication of ideas going on - machines obviously do not have ideas. What's really happening is that you arrange the physical world in such a way that it behaves the way you want it to behave. (Let's not forget that program text is physical - a configuration of "bits" in a piece of hardware - which we construct using our fingers at a keyboard; also, that there is no distinction to be made between (formal) language and machine.) The fact that we draw on natural language to define the symbols of our programming languages (the sprockets of our machines) is entirely parasitic on the primitive notion of machine (formal language). We could replace all the keywords of our favorite language by random strings, and programming would still be possible in principle; that's trivially obvious, but it is not trivial. It means that human communication is not a primitive element of programs. Yet for obvious practical reasons we do prefer to use natural-language facsimiles in our programming languages - facsimiles, since predefined formal strings like "map" or "conj" only look like natural language words. And when we define local names we invariably use natural-language-like terms. The possibility of doing this is obviously very useful, but it remains secondary to the primitive structure of mindless symbol manipulation machinery. Choosing a good natural-language representation for a variable or function name involves finding a word or phrase that *expresses* the computational semantics, and *that* is where human communication enters the picture: the expressiveness of the programming code itself, as a hybrid of quasi-natural-language piggybacking on abstract formal language. This suggests an alternative to LP "methodology": programming not as literary composition but as language design. Program texts to be viewed not as essays expressed in a programming language, but as extensions of the language. For example, every function definition of arity > 0 effectively defines a mini-DSL as an extension of the base language. The parameters to the function are "terms" that are locally (temporarily) added to the lexicon. Ditto for "def"ed names. Lambda expressions ("anonymous" functions) are themselves names - you can view them as one-off names, or you can think of the language lexicon as already implicitly containing all possible lambda expressions as names. If your language has hygienic macros like Clojure you can extend the syntax as well. A "good" language extension (program) is one that uses the expressive resources of a natural language to extend the vocabulary and/or syntax of the base language in "good" ways - good involving principles of clarity, parsimony, etc. On this view documentation is entirely external to programming. In any case, the result is a radically different view of what programming is all about: programmers not as mere users of languages designed by others, but as designers of (derived) languages, whose task is to extend programming languages in ways that fit (express) the problem domain. Every program is then viewed as a kind of DSL design. Libraries are DSLs that can be used in other designs. Locals are not "variables", which puts the focus on semantics, but "lexical extensions", which puts the focus on language and thus expressiveness. Etc. Since programming language designers are generally held to a higher standard with respect to clarity and expressiveness - you'd better put some serious thought into syntax and predefined names if you're going to design a new language - programmers who think of themselves as language designers might do a better job of writing understandable programs. And from this perspective documentation is a completely separate issue that should not be confused with writing code. (I am NOT saying that documentation is unimportant, only that it is not to be conflated with programming.) So I would modify Knuth's plea to something like ""Instead of imagining that our main task is to instruct a *computer* what to do, or to explain to *human beings* what we want a computer to do, let us concentrate rather on designing *expressive* languages that solve our problems." Needs work I guess, but you get the idea. Improve programming. Also improve documentation. Don't confuse the two. > Discussions seem to get lost in a debate about the machinery rather > than the goal. We focus our debate on docstrings versus markup versus > wiki. We consider literate programming to be "too much machinery". > Not too much, but the wrong kind of machinery. > > In these forums there is rarely find any example of "present-at-hand" > issues of communication. That is, given a large program (e.g. Axiom, > Clojure), what is it that we need to communicate, to what audience, > and at what level of detail? > Or, for what purpose? The author cannot determine who reads his or her work, but can decide on a purpose. You might write a book with a college freshman audience in mind, but to me that translates into "for the purpose of instructing people with a certain level of knowledge and experience", which frees it from any particular social group. Even though I've challenged some of your assumptions I think you are right to stress this aspect of program texts and documentation - the purpose (or intended audience, or whatever we want to call it). Lots of comments on behalf of LP ignore this; a common ploy is to offer an expressly pedagogical LP text as an example that generalizes to programming of all kinds. But different purposes (audiences) demand different approaches. (I've sketched out some ideas along these lines at https://github.com/mobileink/codegenres/wiki/Code-Lit.) The presentational output of LP is great for some purposes, so LP may be a good method; but note that that does not imply that the source from which such a presentational form is generated must be an LP text. > > Axiom focuses on "The 30 year horizon" under the assumption that the > computational mathematics will be forever valid and that the audience > will be unable to contact the likely-dead authors. > > Pharr and Humphreys' "Physically Base Rendering" [0] is written as a > literate program, a book that won an Academy Award, using Tex and > C++. The very first thing they mention in the preface is the > "Audience". They communicate to humans and, also, to machines. > The goal of the book is explicitly pedagogical, so LP is useful. But it doesn't generalize to all purposes. PBR is clearly becoming the poster child for LP. It won an award! LP good! (I'm not saying that's what you mean here. I'm just sayin'.) But what won the award was the book - the explanation, presentation, and content of the book/program, not the LP source text of the book/program. The award was for "for their formalization and reference implementation of the concepts behind physically based rendering<http://www.oscars.org/press/pressreleases/2014/20140108.html>" - not for using LP. In principle the same book could have been produced by other methods. > > What is the audience for Clojure? > I don't think it has an "audience" - it's a tool, not a text. But it does have markets - programmers and other users of such tools. > > Common Lisp has achieved a long-term horizon by raising the language > to a standard. No standard is perfect but it does make it possible to > construct programs which have a stable base for communication. That > base makes it possible to write a book like "Lisp in Small Pieces" [1] > which communicates ideas in natural language using an embedded program > as a reduction to practice. > A great book, but it's not really about CL in particular. It's mostly Scheme, and it's not LP. So I'm not sure why you're citing it - is your point that it exemplifies narrative prose with embedded code? Maybe I'm not following your point about communication here. > So the fundamental point is what to communicate to what audience, > not how to implement it. Different audiences will need different > implementations (e.g. docstrings for REPL users) but we must avoid > losing ourselves in the noise. > You can say that again. Another major source of noise is failure to distinguish between the presentational output of classic LP and the use of LP tools and techniques. There's more than one way to get that sort of output. > > Axiom, by choice, has a defined audience. Does Clojure? > I'm not sure what you're getting at with this question. What kind of answer would count as a good one, for you? Cheers, Gregg -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
