Hi there !! Sometime ago I tried some patches and external scripts to use lyx for literate programming, and I had used it for some reasonable sized projects already. I decided that it was a successful experience, so I re-coded the patches and incorporated the scripts within lyx, trying to preserve the guidelines of the lyx project as much as I could (and understood). I would like to submit these patches (against lyx-1.0.0pre4) for your appreciation. I hope the patches don't clash with any existing development direction. I really enjoy using LyX as a noweb->latex->html programming tool, and I am sure other people might find the same. Moreover, I divided the modifications in several patches. If you don't get to a conclusion (to absorb all these modifications into lyx or not), maybe you could apply the harmless ones and leave the others as an "optional" package... Thanks. Edmar W. Jr. *-*-*-*-*-* INTRODUCTION *-*-*-*-*-* In a glance, literate programming is a way to put text and code in the same file. The tool I use to separate them and produce the final code and documentation is the "noweb" one (there are many others: cweb, nuweb, etc.). The noweb web page is: http://www.cs.virginia.edu/~nr/noweb/intro.html Here is a tiny example: -----------begin of example----------------- \begin{document} Hello there, this is the documentation of my little program. It depicts my program divided in three major steps: \section{Read data} <<First step>>= scanf ("%d", &buffer); @ \section{Compute results} <<Second step>>= result += buffer; @ \section{Output results} <<Third step>>= printf("%d", result); @ Now I can put everything together in one single segment, noweb will take care of generating the resulting file for me: <<program.c>>= void main (int argc, char *argv) { int result, buffer; <<First step>> <<Second step>> <<Third step>> } @ \end{document} -----------end of example------------------- *-*-*-* ROADMAP *-*-*-* 1 - The first patch is to allow the choice of the color of the new-line character. As you will see later, I make extensive use of this character when I am editing code, and at the way lyx is today it stands out too much, distracting me from the code itself. With this option, I can chose a color that makes it almost disappear in the background. Obs: The default color is the same it is today. 2 - Patch to include new shortcut buttons in the toolbar. Using lyx for literate programming requires a lot of layout changes, like section->standard->scrap->standard->section, etc.. So I added some buttons to the toolbar. The patch inserts new xpm figures and does the proper changes in "toolbar.C". Obs: The customization of the toolbar itself is left to the user do in his/her lyxrc file. Thus, the toolbar defaults to its present looks. 3 - Patch to not output the protected space '~' during translation of latex paragraphs. It seems that latex does not complain if you put '~' characters in place of regular ' ', even within latex commands. But I am using the latex output to generate a C program. For that reason, I must get ride of those useless protected spaces. Obs: I don't know if this was there in the lyx code as a feature or more like a harmless bug. The user shouldn't perceive any change. 4 - Patch to always produce "nice-latex". Motivation: ----------- Noweb takes the output file (latex) produced by lyx and create a new one with all the web stuff added on. Note that noweb takes care of *not* increasing the number of lines of the original program. That means, if I have a latex error, lyx is still able to point to the exactly point in the document where the error occurred. As a side effect, noweb will produce a file, in this case named "program.c" with all the programming code written on it *plus* some "#line 999# directives. Those line directives are very helpful when debugging, because it directs gdb to display the latex file (on the gdb screen) at the correct line number, instead of displaying "program.c" which contains no comments, nor text, etc. Unfortunately, when we ask lyx to run latex, it will save the file in "usual" mode. This is bad for debugging, because the text will look really ugly, and the pieces of code will be spaced away. The work around I used for a long time was the following: 1 - run latex to produce the .dvi (actually it also produces the program.c, but I ignore it at this point) 2 - Export "nice latex" 3 - run a make file that runs again noweb and produces a new program.c Later when I am debugging program.c, the debugger will load the "nice latex" file and will display the cursor at the "#line 999" found on program.c. The patch: ---------- With this patch, the 3 steps above are eliminated. But I must bring up some points for discussion: First, the precision to locate latex errors will be smaller, i.e., within lines, instead of within words. I think we can leave with that, and I give you two reasons: 1 - The error messages contains one full line of text detailing where the error is. 2 - Since lyx generates most of the latex commands itself, latex error are rare. Second, by getting rid of the \batchmode command, the latex process may get stuck waiting for some input from the keyboard. (whenever a error is found). The solution I found is to append a '<&-' redirection at the end of the latex command line. This redirection is equivalent to type ctrl-D when latex ask for some keyboard input. The ksh, sh, and bash shells all have this feature and all have the same syntax. But just in case, I thought it would be better to make it completely shell independent by having a default entry and allowing the user specify a different termination in the lyxrc file. The command \latex_command_input was introduced for that purpose. command: default: \latex_command_input <&- 5 - Patch to allow Item_Environment and Environment layouts to have a "dummy" latex command name. This makes lyx to treat the layout as an Environment (and have the screen behavior associate with that) but not generate any latex command (i.e., no \begin{} and \end{}, nor \item on Item_Environments. Obs: This patch is completely transparent to users. Presently lyx would generate \begin{dummy}, \end{dummy} for an environment whose latex name were defined as "dummy" (which is completely useless anyway...). 6 - The addition of the Scrap layout to lyxmacros.inc. This type of layout can be virtually be present in any type of document. It is defined as a dummy Item_Environment. Obs: The user will see one more option in the pop-up list of layouts: "Scrap". I don't thing this should bring any conflict or discussion but you may want to raise two questions: 1 - Is there a better name ? 2 - Should we restrict the scraps to be on "this" and "that" types of documents ? IMHO: 1 - I did not choose this name. "Scrap" is the name used in the documentation of the "nuweb" package (it is not misspelled, nuweb and noweb are different things), which I first used in my career, and I got used to. However, noweb is the package I use today. They prefer to use the term "Chunk".... 2 - Every literate program I have seen (mine or not) either used book, report or article as document type. It is possible though, to insert code in any kind of document. I don't see any advantage on restricting the user, so I put the scrap layout in lyxmacros.inc 7 - Patch to have different buttons and menu options for running latex, and building the code. The following new entries in the lyxrc are possible now: command: default: \web_command noweave -delay -index \web_extension .nw \web_error_filter cat \build_command make \build_error_filter cat When the "build program" menu option is chosen or the corresponding button in the toolbar is pressed, a nice latex file is generated, the only difference is that the file extension will not ".tex" but the one specified by \web_extension. Then lyx invokes \build_command and \build_error_filter. This filter is to allow lyx to identify web/compiler errors independently of the web/compiler chosen. (As an example, the last attachment is the C program I use for filter). In case of any errors, the error boxes are inserted with messages from the output of the error filters using the existing lyx mechanism for that. This is kind of cool, since now I can see the compilation errors with lyx!!! When the "run latex" menu option is chosen, a nice latex file is generated. If there is any paragraph of type Scrap, the file is generated with extension \web_extension, and piped through \web_command and \web_error_filter. The \web_command must generate a file with extension .tex. If no Scrap is present, then lyx generates the same nice latex file but with extension .tex. Finally, latex is invoked and regular processing continues as it is today. In summary, the "build program" function is pretty much like the "run latex" one but involving different commands. See schematic: "run latex" on plain document: -> generate .tex -> invoke latex -> show errors (today behavior) "run latex" on document with scraps: -> generate same -> invoke \web_command -> invoke latex -> show latex errors file above but that generates the \______________ -> show web errors renamed to .nw .tex file (line numbers are not changed !!) "build program" on any kind of document: -> generate same -> invoke \build_command -> show compilation errors file above but that generates .c .h files renamed to .nw compiles program, etc.. 8 - Patch to implement "server-goto-file-row" function. When debugging code, either with emacs/gdb or ddd/gdb, it is possible to invoke the editor at the current execution position with a single key stroke. To compensate the fact that lyx cannot be integrated into these tools I defined the editor invocation key sequence on the ddd to execute: echo "LYXCMD:monitor:server-goto-file-row:@file@ @line@" >~/.lyxpipe.in and implemented this function on lyx. Now, when I find a bug, and I want to change the code at that location, I press the hot key on ddd that executes the "echo" above and voila': lyx jumps directly to the debugging point I was working and centers the cursor in the same line ddd was pointing to. Obs: This patch should be transparent to users that will not use literate programming. *-*-* TODO *-*-* 9 - Patch to implement a batch mode. I need to be able to run a script that extracts a collection of lyx files from a CVS repository, generates the code, compiles it, and runs it. I understand I can use the lyxserver to load and export a file. But in that case the process may not have a display !!. Thus, I need some way to execute some lyx commands from switches in the command line and at the same time, avoiding all X11 function calls.
-- /*----------------------------------------------------------------------*/ /* Edmar Wienskoski Jr. - [EMAIL PROTECTED] */ /* - http://www.cs.rice.edu/~wiensk */ /*----------------------------------------------------------------------*/ ____ | [] | ______()_||_ ---+----+--- ------------ ------------ ------------ | [] | | | | | | | | | | | ___| | |_|______|_| |__________| |__________| |__________| |______________\ "o-o o-o""o-o o-o""o-o o-o""o-o o-o""o-o O-O-O o-o "
edmar-3-prot_space-981216.patch
edmar-5-dummy_env-981216.patch
#include <stdio.h>
#include <strings.h>
char buffer[200][200];
int last_buf_line;
int last_err_line;
int err_line;
void
output_error (int buf_size, int error_line, char *tool)
{
int i;
fprintf(stdout, "! Build Error: ==> %s ==>\n", tool);
for (i=0; i<buf_size; i++)
fprintf(stdout, "%s", buffer[i]);
fprintf(stdout, " ...\n\nl.%d ...\n\n", error_line);
}
char *noweb_msgs[] = {
"couldn't open file",
"couldn't open temporary file",
"error writing temporary file",
"ill-formed option",
"unknown option",
"Bad format sequence",
"Can't open output file",
"Can't open temporary file",
"Capacity exceeded:",
"Ignoring unknown option -",
"This can't happen:",
"non-numeric line number in" };
int
noweb_try (int buf_line)
{
char *s, *b;
int i;
b = buffer[buf_line];
/* First type is lines with "...<<...>>..." */
s = strstr(b, "<<");
if (s != NULL) {
s = strstr(s+2, ">>");
if (s != NULL)
return 1;
} else {
for (i=0; i<12; i++) {
s = strstr (b, noweb_msgs[i]);
if (s != NULL)
break;
}
if (s != NULL)
return 1;
}
return 0;
}
void
noweb_scan (void)
{
last_buf_line = 0;
while (fgets(buffer[0], 200, stdin)) {
if (noweb_try(0))
output_error(1, 0, "noweb");
}
}
int
xlc_try (int buf_line)
{
char *s, *t;
t = buffer[buf_line];
s = t+1;
while (*s != '"' && *s != ' ' && *s != '\0')
s++;
if (*t != '"' || *s != '"' || strncmp(s+1, ", line ", 7) != 0)
return 0;
s += 8;
err_line = atoi(s);
return 1;
}
void
xlc_scan (void)
{
last_buf_line = 0;
while (fgets(buffer[last_buf_line], 200, stdin)) {
if (xlc_try(0))
output_error(1, err_line, "xlc");
}
}
void
aix_scan (void)
{
last_buf_line = 0;
while (fgets(buffer[0], 200, stdin)) {
if (noweb_try(0))
output_error(1, 0, "noweb");
else if (xlc_try(0))
output_error(1, err_line, "xlc");
}
}
void
discharge_buffer (int save_last)
{
if (last_err_line != 0) {
if (save_last != 0) {
output_error(last_buf_line-1, last_err_line, "gcc");
strcpy (buffer[0], buffer[last_buf_line-1]);
last_err_line = err_line;
last_buf_line = 1;
} else {
output_error (last_buf_line, last_err_line, "gcc");
last_err_line = 0;
last_buf_line = 0;
}
}
}
void
gcc_scan (void)
{
char *s, *t;
last_buf_line = 0;
while (fgets(buffer[last_buf_line], 200, stdin)) {
/****** Skip lines until I find an error */
s = strpbrk(buffer[last_buf_line], " :");
if (s == NULL || *s == ' ')
continue;
/****** Must find a ":999:" or a space, same number is continuation */
do {
/****** Search first ":" in the error number */
s = strpbrk(buffer[last_buf_line], " :");
last_buf_line++;
if (s == NULL || *s == ' ') {
err_line = 0;
discharge_buffer(1);
continue;
}
/****** Search second ":" in the error number */
t = strpbrk(s+1, " :");
if (t == NULL || *t == ' ') {
err_line = 0;
discharge_buffer(1);
continue;
}
/****** Verify if is all digits between ":" */
if (t != s+1+strspn(s+1, "0123456789")) {
err_line = 0;
discharge_buffer(1);
continue;
}
/****** OK It is an error, get line number */
err_line = atoi(s+1);
if (last_err_line == 0 || last_err_line == err_line) {
last_err_line = err_line;
continue;
}
discharge_buffer(1);
break;
} while (fgets(buffer[last_buf_line], 200, stdin));
}
discharge_buffer(0);
}
void
build_scan (void)
{
char *s, *t;
last_buf_line = 0;
while (fgets(buffer[last_buf_line], 200, stdin)) {
/****** Skip lines until I find an error */
if (last_buf_line == 0 && noweb_try(0)) {
output_error(1, 0, "noweb");
continue;
}
s = strpbrk(buffer[last_buf_line], " :");
if (s == NULL || *s == ' ')
continue;
/****** Must find a ":999:" or a space, same number is continuation */
do {
/****** Search first ":" in the error number */
s = strpbrk(buffer[last_buf_line], " :");
last_buf_line++;
if (s == NULL || *s == ' ') {
err_line = 0;
discharge_buffer(1);
continue;
}
/****** Search second ":" in the error number */
t = strpbrk(s+1, " :");
if (t == NULL || *t == ' ') {
err_line = 0;
discharge_buffer(1);
continue;
}
/****** Verify if is all digits between ":" */
if (t != s+1+strspn(s+1, "0123456789")) {
err_line = 0;
discharge_buffer(1);
continue;
}
/****** OK It is an error, get line number */
err_line = atoi(s+1);
if (last_err_line == 0 || last_err_line == err_line) {
last_err_line = err_line;
continue;
}
discharge_buffer(1);
break;
} while (fgets(buffer[last_buf_line], 200, stdin));
}
discharge_buffer(0);
}
int
main (int argc, char **argv)
{
if (argc == 2) {
switch (argv[1][0]) {
case 'n': /* noweb */
noweb_scan();
break;
case 'g': /* gcc compiler */
gcc_scan();
break;
case 'x': /* IBM xlc compiler */
xlc_scan();
break;
case 'S': /* Sun C compiler */
break;
case 'a': /* AIX system, scans for both noweb and xlc */
aix_scan();
break;
case 's': /* Solaris system, scans for both noweb and gcc */
case 'b': /* build, scans for both noweb and gcc */
build_scan();
break;
default:
gcc_scan();
break;
}
} else {
gcc_scan();
}
}
