Why Literate Programming?
In my life as an artchitect, I find that the single thing which inhibits young professionals, new students most severely, is their acceptance of standards that are too low. If I ask a student whether her design is as good as Chartres, she often smiles tolerantly at me as if to say, "Of course not, that isn't what I am trying to do...I could never do that."
Then, I express my disagreement, and tell her: "That standard must be our standard. If you are going to be a builder, no other standard is worthwhile."
--Christopher Alexander, May 1996 in (Patterns of Software, Gabriel)
The best programming language is English. Everything else is notation.
-- Timothy Daly (2011)
Here is an example literate program in HTML
Here is a Journal article about Literate Programming and Reproducible Research
I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title "Literate Programming"
Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.
-- Donald Knuth "Literate Programming (1984)" in Literate Programming CSLI, p99
Step away from the machine. Literate programming has nothing to do with tools or style. It has very little to do with programming. One of the hard transitions to literate programming is "literate thinking".
-- Timothy Daly (2010) in Lambda the Ultimate forum [LTW10]
The hardest part of literate programming is the documentation.
--Timothy Daly (2011)
The effect of this simple shift of emphasis can be so profound as to change one's whole approach to programming. Under the literate programming paradigm, the central activity of programming becomes that of conveying meaning to other intelligent beings rather than merely convincing the computer to behave in a particular way. It is the difference between performing and exposing a magic trick.
--Ross Williams, FunnelWeb Tutorial Manual, pg 4.
Another thing I've been enjoying lately is literate programming. Amazingly it turns out to be faster to write a literate program than an ordinary program because debugging takes almost no time.
--Bill Hart, SAGE Mailing list, May 3, 2010
The conversation is much more direct if the Design Concept per se, rather than derivative representatives or partial details, is the focus.
--Fred Brooks: The Design of Design: Essays from a Computer Scientist.
We are banning the old notion of literate programming that I used when developing TeX82 because documentation has proven to be too much of a pain.
--Donald Knuth TUG 2010
And on Axiom's "30 Year Horizon" focus:
Once upon a time I took great care to ensure that TeX82 would be truly archival so that results obtainable today would produce the same output 50 years from now but that was manifestly foolish. Let's face it, who is going to care one whit for what I do today after even 5 years have elapsed, let alone 50. Life is too short to re-read anything anymore in the internet age. Nothing over 30 months old is trustworthy or interesting.
--Donald Knuth TUG 2010
From a November, 2011 Knuth interview:
Yet to me, literate programming is certainly the most important thing that came out of the TeX project. Not only has it enabled me to write and maintain programs faster and more reliably than ever before, and been one of my greatest sources of joy since the 1980s -- it has actually been indispensable at times. Some of my major programs, such as the MMIX meta-simulator, could not have been written with any other methodology that I've ever heard of. The complexity was simply too daunting for my limited brain to handle; without literate programming, the whole enterprise would have flopped miserably.
If people discover nice ways to use the newfangled multithreaded machines, I would expect the discovery to come from people who routinely use literate programming. Literate programming is what you need to rise above the ordinary level of achievement. But I don't believe in forcing ideas on anybody.
-- Knuth, Donald Interview with Donald Knuth
An abbot of the Seven-Clawed Blind Eagle Clan was tallying the deliverables of the temple monks, when he noticed that a monk of the Laughing Monkey Clan had produced no design documents.
"What if a problem were discovered in our production system?" the abbot asked the monk. "How would the cause be understood by any besides yourself?"
"The code itself should be examined," said the monk. "The Document is a sickly beast, easily subject to the Three Plagues of Error: omission, obfuscation, and obsolescence."
The abbot reported this to the Java master, who said: "Have the monk balance on one foot with his staff outstretched, every day from dusk to midnight. If he can say a word to please me I will revoke the punishment, but not otherwise."
"Is there such a word?" asked the abbot.
"It is difficult to be certain," reflected the master. "Yesterday I was pleased by the sound of a cricket chirping after the first three drops of rain. The day before, the plashing of milk in a pail returned me to a pleasant memory of my youth. Perhaps the answer lies in the fragrance of a white lotus that drifted once in a pond below my window."
--Qi The Codeless Code
Once a program has been developed and the developers have moved on to other tasks it needs to be maintained. The fundamental problem is that if you modify code you didn't write, you don't see the big picture and you don't understand the reasons why the code is written the way it is. Thus changing code without a larger context is almost always going to introduce new bugs.
The only way to correctly change code is to deeply understand the implications of the change. This requires a deep understanding of the code and an awareness of the big picture. Yet the "why" of code is rarely ever written down in standard programming practice. The goal is only to elaborate the "how" so the machine can perform the task. The programmer communicates with the machine.
Literate programming, as used in Axiom, is an attempt to communicate with other users, developers, and researchers in addition to the machine. The goal is to have the program read like a story so that others can understand the rational, the theory, the choices, the implications, and the implementation context as well as the "how".
This code is intended to live forever but it is highly probable that you will not. Write to communicate with the next person to pick up the torch. When you explore code, write down what you learn. When you change code, explain why you made your choices. When you write new code explain what others need to know to maintain it.
The Axiom system is gradually being documented in a set of volumes. These change with every update to the system since they contain the actual system source code. The volumes listed here are updated every other month when the system is distributed. The current volume set is:
- Combined Table of Contents This is the table of contents from the existing volumes combined into one document for easy reference.
- Volume 0: Axiom Jenks and Sutor This is the reconstructed Jenks and Sutor volume.
- Volume 1: Axiom Tutorial This is the tutorial volume ISBN 1-411-66587-X. Hardcopy is available from Amazon.com or Lulu.com
- Volume 2: Axiom Users Guide This is a more detailed explanation with current information for Axiom users.
- Volume 3: Axiom Programmers Guide This is information about the language and algebra hierarchy for Spad language programmers.
- Volume 4: Axiom Developers Guide This is a collection of useful information for developers.
- Volume 5: Axiom Interpreter This is the source code and explanation for the interpreter.
- Volume 6: Axiom Command This covers the axiom commands, sman, and some other system related issues.
- Volume 7: Axiom Hyperdoc This is the source and explanation of the X11 hyperdoc subsystem.
- Volume 7.1: Axiom Hyperdoc Pages This is the source and pages for Hyperdoc.
- Volume 8: Axiom Graphics This is the source and explanation of the X11 graphics subsystem.
- Volume 9: Axiom Compiler This is the source and explanation of the spad compiler.
- Volume 10: Axiom Algebra Implementation This is a multi-volume set covering the algebra. The first volume deals with implementation issues.
- Volume 10.1: Axiom Algebra Theory This volume gives background theory for various algebra topics.
- Volume 10.2: Axiom Algebra Categories This is the source code for all of the categories.
- Volume 10.3: Axiom Algebra Domains This is the source code for all of the domains.
- Volume 10.4: Axiom Algebra Packages This is the source code for all of the packages.
- Volume 10.5: Axiom Algebra Numerics This is the source code for all of the numerics.
- Volume 11: Axiom Browser This is the source and explanation of the new Firefox browser front end.
- Volume 12: Axiom Crystal This is the design documents and internals for the crystal interface.
- Axiom Bibliography This contains all the literature references to, and from, Axiom.
Axiom is being reworked to use the Firefox browser as the new front end. Static pages from the new hyperdoc show some of the details. These pages will be "live" in the new Axiom hyperdoc.
The Axiom build graph shows the algebra hierarchy in required build order. Note that the edges in the graph only show dependence on the prior level in the hierarchy. There are approximate 1100 total nodes. Each node represents a single domain, category, or package. Graph nodes that are elliptical show objects that get build as part of the bootstrap process.
The algebra graph in its current form is available. You'll need a reasonably new browser to see the graph. This is a work in progress.