login  home  contents  what's new  discussion  bug reports     help  links  subscribe  changes  refresh  edit

This page discusses the pros and cons of a change in pamphlet syntax away from the currently (23-Jul-2007) used noweb syntax.

Please add your pros and cons or any other notes into the text. Try to state facts, not opinions. If you feel that you must present your opinion, attribute it by your name.

Syntax Proposal for Pamphlets

  1. Noweb style (http://www.eecs.harvard.edu/~nr/noweb/)

    One can use '[[some text]]' to reference chunks but it is basically (ab)used to set verbatim text.

    Example:

           <<chunkname>>=
             some code
             <<chunk-1>>
             other code
             <<chunk-2>>
           @
    

  2. LaTeX style

    This style is proposed by Tim Daly in order to get rid of the weave step of noweb. The hope is that documentation can be produced faster by LaTeX itself without the need of a weave step. Another hope is that cl-web (written by Cliff Yapp) which would be able to tangle this syntax into an appropriate code file integrates nicely into the Axiom LISP environment.

    Example:

           \begin{chunk}{chunkname}
             some code
             \chunk{chunk-1}
             other code
             \chunk{chunk-2}
           \end{chunk}
    

  3. AxWeb style

    Propose by Stephen Wilson. The idea is similar to have a noweb replacement entirely written in LISP so that it integrates nicely with the rest of Axiom. The syntax should be different from LaTeX because, LaTeX files are not Pamphlets.

    There is a minimal wiki page with source code and documentation. (see the AxWeb page).

    References in text are made via @<some text@>.

    Example:

           @<chunkname@>=
             some code
             @<chunk-1@>
             other code
             @<chunk-2@>
           @
    

  4. LEO (http://webpages.charter.net/edreamleo/front.html)

    LEO is a tool that supports different views on the same data (a .leo file). LEO files are human readable files and could in some sense be seen as pamphlets, but they use a non-noweb syntax. The syntax, however, does not matter so much since editing would be done with the LEO editor.

  5. OldTeXmacs (http://texmacs.org/)

    OldTeXmacs is a tool with a builtin rendering machine. It allows to enter data in a wysiwyg fashion. OldTeXmacs uses its own file format which is human readable, but uses a non-noweb syntax. The syntax, however, does not matter so much since editing would be done with the OldTeXmacs editor.

  6. LyX (http://www.lyx.org/)

    Lyx is a editor with a lot of LaTeX support. There is also some mild support of literate programming through noweb. Lyx uses its own file format .lyx which is human readable looks similar to a TeX (not LaTeX) syntax. The syntax, however, does not matter so much since editing would be done with the Lyx editor.

Recent Discussion

Pros and Cons

  1. Noweb style

    Pros:

    • noweb is an established external tool that works reasonably well and it works now
    • current build mechanism uses it
    • noweb offers some features like creating hyperlinks inside code chunks

    Cons:

    • noweb is said to be slow (Ralf: I wouldn't confirm it.)
    • noweb is not be default installed on every system
    • due to a bug one must escape << in places that would not require an escape
    • starting a documentation chunk right after "@ " is confusing for a LaTeX writer. The newline after "@ " should not be significant and "@ " should rather behave like ordinary LaTeX environments.

      (steve: noweb allows documentation to follow an @ sign and space, allowing one to choose not to emit the newline. Same as Axweb.)

    • Because noweb is not written in Lisp it does not integrate as smoothly as possible with a Lisp based Axiom (Cliff: This may not be an issue for some people.)
  2. LaTeX style

    Pros:

    • No need to learn new syntax.
    • Tools like mmm-mode for Emacs can probably be easily adopted to different markers.

    Cons:

    • The fact that a pamphlet contains code is physically not as obvious as with different syntax.
    • It is still necessary to learn that \begin{chunk}{chunkname} ... \end{chunk} is a special environment.
    • Current tools like the mmm-mode do not support a non-noweb syntax out of the box. Modification of such tools necessary --> Who would distribute such modifications?
    • \newenvironment{chunk}{...}{...} defines a command \chunk. Basically \begin{chunk} will be translated to execute \chunk. Thus, it would be hard to actually use \chunk also as a name for referencing other code chunks. \use or \usechunk would be workarounds.
    • \chunk, \use, or \usechunk are too long. Would be better to define a escape mechanism (see listings package) so that references could be made in a more readable way.
    • problem with code already demonstrated at http://lists.nongnu.org/archive/html/axiom-developer/2006-10/msg00438.html The problem with fixing '\', {, and } is that code cannot look natural and cannot simply be pasted into a pamphlet without a human adjusting it.

      Example:

                 \begin{chunk}{chunkname}
                   b: Boolean := true;
                   for i in mylist repeat {
                     \usechunk{anotherchunk}
                     b := b/\odd? i;
                   }
                 \end{chunk}
      

  3. Axweb style

    Pros:

    • Most probably the markup will not appear in any programming language and thus not interfere with the code. "<<" has a higher probability of collision. (steve: almost true. I have reserved the sequence `@(' for future use, which is relatively common in Lisp macro definitions.)
    • Fast? (Ralf: I haven't checked. Cliff: Benchmarking is still on the TODO list. Steve: Looks about twice as fast as noweb according to local experiments)
    • Very flexible - intended to support advanced experiments with Literate Programming concepts.

    Cons:

    • Strings could contain the sequence @<blah@> and would require the need to escape.
    • The only program that understands this syntax is written in LISP by Stephen Wilson. It therefore binds people who want to use this syntax to installing LISP. (Cliff: It is unclear if noweb could be made to read this syntax in its more "normal" uses - I suspect it can be customized to do so, up to a point. If incompatibilities appear at a fundamental level it will most likely be due to axweb doing things noweb wasn't designed for.)
    • People are required to learn a new non-standard language. (In contrast to noweb-syntax it would be Axiom-only specific.)
  4. LEO

    Pros:

    • A more modern view on literate programming. LEO files could be (partly) seen as the Crystal and the LEO editor would allow to see different faces.

    Cons:

    • LEO is too code oriented and does not favor documentation over code.
    • The editor is not as powerful as emacs.
  5. OldTeXmacs

    Pros:

    • Wysiwyg-editing removes the need of a weave step altogether, since that is done while typing.
    • supports Axiom sessions and sessions of other CAS

    Cons:

    • OldTeXmacs does not support literate programming. There would be need to create special code environments and a tangle mechanism.
    • OldTeXmacs is maintained by a very small community
    • Uses its own file format and thus binds people to the OldTeXmacs editor
    • Ralf: OldTeXmacs is hard to learn and crashed on me several times.
    • Cliff: OldTeXmacs does not currently have an up-to-date native Windows port.
  6. LyX

    Pros:

    • very active developers (according to the lyx-dev mailing list)
    • Does have a native version for Windows.
    • some support for literate programming (Ralf: I haven't really looked into it.)

    Cons:

    • Uses yet another native file format and binds people to the LyX editor

Should a Pamphlet be understood by TeX?

Since TeX is a programming language, the:

    <<chunkname>>=
      some code
      <<reference to a chunk>>
    @

syntax can probably be made to be directly understood by TeX itself. All that is needed is a clever TeX programmer who can do it.

Pros:

  • No weave step
  • No syntax change necessary

Cons:

  • Automatic embellishing of code with hyperlinks is hard to program in TeX;
  • a compiler for the appropriate language should be used instead of writing a parser in TeX
  • TeX might take longer due to processing of uncommon syntax (depends on the cleverness of the implementation)
  • Extensions beyond the basic << >>= @ arrangement would also have to be coded in TeX.

How to embellish code chunks?

Code chunks should almost look like real code except for the places where there is a reference to another code chunk. Syntax highlighting and hyperlinking should be done during a weave step.

Several levels of embellishment. The more features is added the more time it costs to analyse the source files. Thus build time gets longer with nicer looking documentation.

  1. If the pamphlet syntax (whatever it is) is (through a style file) understood directly by TeX, then code comes out like verbatim text.
  2. In first approximation only a few enhancements to the code part (for example as demonstrated by ALLPROSE in connection with noweb) would be made.
  3. The full power of a compiler for the corresponding language is used to embellish the code with hyperlinks.

This should be configurable at build time - since the weave step itself produces the more advanced markups, there should be a configure option to select how fancy the generated documentation is.

Further Notes

  • code pasted to code chunks should not need any escape characters (that should be controlled from outside, see LaTeX's listing package)
  • syntax change and tool development should happen in a branch
  • A literate programming tool should best not be too tightly integrated into Axiom since it might be interesting to other projects.
    • Maybe it should become an external project with lots lots of input from Axiom developers.
    • People working on such a tool need not be full fledged mathematicians. (Ralf: I expect more developers if we rather support external projects like noweb3.)
    • It should be a standalone tool that can be called from anywhere (LISP, C, Aldor) and not just LISP alone. (Cliff: Multiple tools are possible, standalone and integrated.)
    • (Cliff: The question of literate programming tools being stand-alone is not an all or nothing design decision - given a chosen syntax, any number of tools can support it. Tool choice can be selected at build time if it remains a contentious issue.)
  • use fancyvrb or listings to typeset code chunks (a non-LaTeX weave step could relax need for the fancyvrb or listings package)
  • the mmm-mode for emacs can be adjusted for different languages in one pamphlet file (Ralf: I just don't exactly know how.) (Cliff: Barring major changes in Emacs 22, the syntax highlighting logic may make this a challenge in Emacs.)
  • indentation of generated code

    The -L option of notangle spits out code without a modification of the indentation inside the pamphlet file. That is disastruous to SPAD or Python, but has its advantagous for Aldor. A compiler that shows also the columns of an error would otherwise be wrong by the indentation amount. Note that the compiler should point into the .pamphlet and not into the generated .as file.

  • Ralf: Maybe the following is a very personal opinion.

    The '[[...]]' syntax is overloading!!! It is mostly (ab)used to typeset verbatim text. There should be something like:

          \defineterm{...}   -- For definition and use of certain terms (as defined in a
          \useterm{...}      -- \begin{definition} ... \end{definition} environment.
                             -- Automatic index and hyperlink generation.
          \usemaketarget{...} -- references into Makefiles
          \adtype{Integer}     -- reference to an Aldor Type
          \adname[SomeType]{somefunction:(%,%)->Integer}     -- reference to an exported Aldor function
    

  • learning curve for begin{chunk}{chunkname} and '<<...>>=' is almost equal
  • If a syntax change is going to happen, then we must have tools ready before the change.

axweb updates --M. Edward (Ed) Borasky, Sat, 28 Jul 2007 20:33:41 -0500 reply
Ed Borasky, 28 July 2007:

R has something like this built into the base distribution. It's called "Sweave", although it tangles as well. The R community is very big on literate programming and what they call "Reproducible Research" -- distributing packages that contain the code and data to create a finished document. Here are a couple of links to start with:

http://www.ci.tuwien.ac.at/~leisch/Sweave/FAQ.html http://www.stat.umn.edu/~charlie/Sweave/

How R does it --znmeb, Sat, 28 Jul 2007 20:35:18 -0500 reply
Ed Borasky, 28 July 2007:

R has something like this built into the base distribution. It's called "Sweave", although it tangles as well. The R community is very big on literate programming and what they call "Reproducible Research"
distributing packages that contain the code and data to create a finished document. Here are a couple of links to start with:

http://www.ci.tuwien.ac.at/~leisch/Sweave/FAQ.html http://www.stat.umn.edu/~charlie/Sweave/ ::

Ed Borasky, 28 July 2007:

I had a brief flirtation with Leo about a year ago. It's essentially a one-person project with a small devoted band of users. It's an interesting UI, and "opinionated software" both in the good and bad senses of the term. The major drawback of Leo in my opinion is that it is written in Python, a language I don't know and have no other reason to learn. In the end, the learning curve was too steep for me and I gave up on it.

As someone stated, OldTeXmacs allows Axiom sessions to be started and automatically captured into a document. It does not directly support literate programming and is harder to use than LyX. However, if you're an Emacs guru, you'll find it easier to learn than if you aren't. As far as a Windows port is concerned, if you can't get a recent native Windows version, it will run under Cygwin and is I think distributed as part of Cygwin.

LyX is quite a bit easier to use than TexMacs? and does support literate programming. The R folks have hooked it up to their Sweave functionality that I described above, but to get the most out of LyX you'll need noweb anyway. They just released 1.5.0 a few days ago. I've been beta testing it for some time and it is close to becoming my editor of choice. About the only thing it doesn't do is the session initiation and capture that OldTeXmacs does. And there are still some things you'll need to do in shell scripts if you want to automate things. I think LyX is also distributed in Cygwin, and there is a native port as well. The native port requires a whole bunch of dependencies ... IIRC Tcl/Tk, Python, GTK, and of course MikTex?. The installer will download those for you if it finds that you don't have them.

So for my money, I'd go with LyX over TexMacs? and either one of them over Leo. I've got a project I'm chipping away at in my free time called Rameau that integrates R, Ruby, LyX and GiNaC?/CLN. I suppose I could add Axiom or some other "real CAS", but I really wanted to make it a scripting-language/math library tool set rather than an "application" as such. The last time I touched it seriously was about nine months ago, but I'm about to fire it up again. The extended abstract is at http://viewvc.rubyforge.mmmultiworks.com/cgi/viewvc.cgi/Rameau/Rameau.pdf?root=cougar&view=co




  Subject:   Be Bold !!
  ( 13 subscribers )  
Please rate this page: