Re: Modules
() Neil Jerram
() Sat, 29 Jan 2011 23:17:08 +
If the modules are installed, that's true. What if they are not?
[...]
For scripts that use uninstalled modules, then, some kind of
solution is needed; ideally one that works for both 1.8 and
1.9/2.0, allows the code needed to live in a single common
file, rather than duplicated at the top of each script; and
continues to work if the script+module tree as a whole is moved
to a different place in the filesystem.
This is what "module catalogs" from Guile 1.4.x provides.
[verbose explanation follows, skip to end for summary]
Basically, you associate with each element in ‘%load-path’ a
sub-association between module name (list of symbols) and
implementation resolution method. For Guile 1.4.x, there are two
types of modules supported:
- scheme source (text)
- shared object library
Of course the implementation must provide the appropriate
interface (as defined by the support in ‘resolve-module’) for
loading; this system cannot be used for arbitrary scheme source or
shared object libraries.
At load-time, the interpreter consults these catalogs
preferentially, falling back to old-style filesystem groping on
lookup failure. A module (of any form) need not include its
location information.
To see how this helps, i use two small scripts:
st:
#!/bin/sh
exec strace -f -e open "$@" 2>&1 | grep -v o.such.file
sta:
#!/bin/sh
exec strace -f -e open "$@" 2>&1
Here is a run of loading (database postgres-table) [scheme code]
from the Guile-PG build tree. Note that it in turn requires
(database postgres) [shared object library]. Also note ‘-L .’
which is explained further down.
$ cd ~/build/guile-pg/.b
$ st guile -L . -c '(use-modules (database postgres-table))'
open("/home/ttn/local/lib/libguile.so.9", O_RDONLY) = 3
open("/home/ttn/local/lib/libltdl.so.7", O_RDONLY) = 3
open("/etc/ld.so.cache", O_RDONLY) = 3
open("/lib/i686/cmov/libm.so.6", O_RDONLY) = 3
open("/lib/i686/cmov/libdl.so.2", O_RDONLY) = 3
open("/lib/i686/cmov/libc.so.6", O_RDONLY) = 3
open("/home/ttn/local/lib/guile/site/.module-catalog", O_RDONLY) = 3
open("/home/ttn/local/share/guile/site/.module-catalog", O_RDONLY) = 3
open("/home/ttn/local/lib/guile/1.4.1.122/.module-catalog", O_RDONLY) = 3
open("/home/ttn/local/lib/guile/site/init.scm", O_RDONLY) = 3
open("./.module-catalog", O_RDONLY) = 3
open("/home/ttn/build/guile-pg/src/postgres-table.scm", O_RDONLY) = 3
open("/home/ttn/local/lib/guile/1.4.1.122/ice-9/common-list.scm", O_RDONLY) =
4
open("/home/ttn/build/guile-pg/.b/src/.libs/postgres.so.0.0.0", O_RDONLY) = 4
open("/home/ttn/local/lib/libpq.so.3", O_RDONLY) = 4
open("/etc/ld.so.cache", O_RDONLY) = 4
open("/lib/i686/cmov/libcrypt.so.1", O_RDONLY) = 4
open("/lib/i686/cmov/libresolv.so.2", O_RDONLY) = 4
open("/lib/i686/cmov/libnsl.so.1", O_RDONLY) = 4
open("/home/ttn/build/guile-pg/src/postgres-types.scm", O_RDONLY) = 4
open("/home/ttn/build/guile-pg/src/postgres-col-defs.scm", O_RDONLY) = 4
open("/home/ttn/build/guile-pg/src/postgres-qcons.scm", O_RDONLY) = 4
open("/home/ttn/build/guile-pg/src/postgres-resx.scm", O_RDONLY) = 4
If i use ‘sta’, there are many "no such file" messages, but NOT
with the modules. Strictly speaking, open(2) savings proves
improvement only for shared object libraries, so you'll have to
trust me (or confirm by looking at the source code) that the
scheme source modules are treated analogously (i.e., no probing).
$ cd ~/build/guile-pg/.b
$ st guile -L . -c '(use-modules (database postgres-table))' > A
$ sta guile -L . -c '(use-modules (database postgres-table))' > B
$ diff -u A B | uniq
--- A 2011-01-30 10:41:25.0 +0100
+++ B 2011-01-30 10:41:32.0 +0100
@@ -1,8 +1,29 @@
+open("/home/ttn/local/lib/tls/i686/sse2/cmov/libguile.so.9", O_RDONLY) = -1
ENOENT (No such file or directory)
+open("/home/ttn/local/lib/tls/i686/sse2/libguile.so.9", O_RDONLY) = -1
ENOENT (No such file or directory)
+open("/home/ttn/local/lib/tls/i686/cmov/libguile.so.9", O_RDONLY) = -1
ENOENT (No such file or directory)
+open("/home/ttn/local/lib/tls/i686/libguile.so.9", O_RDONLY) = -1 ENOENT (No
such file or directory)
+open("/home/ttn/local/lib/tls/sse2/cmov/libguile.so.9", O_RDONLY) = -1
ENOENT (No such file or directory)
+open("/home/ttn/local/lib/tls/sse2/libguile.so.9", O_RDONLY) = -1 ENOENT (No
such file or directory)
+open("/home/ttn/local/lib/tls/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No
such file or directory)
+open("/home/ttn/local/lib/tls/libguile.so.9", O_RDONLY) = -1 ENOENT (No such
file or directory)
+open("/home/ttn/local/lib/i686/sse2/cmov/libguile.so.9", O_RDONLY) = -1
ENOENT (No such file or directory)
+open("/home/ttn/local/lib/i686/sse2/libguile.so.9", O_RDONLY) = -1 ENOENT
(No such file or directory)
+open("/home/ttn/local/lib/i686/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT
(No such file or directory)
+
Re: Modules
Hi Neil, On Sun 30 Jan 2011 00:17, Neil Jerram writes: > Andy Wingo writes: > >> If you are using modules, they are already in one global namespace, >> the various roots of which are in the %load-path and/or >> %load-compiled-path. (resolve-module '(foo bar)) should work >> regardless of what file calls it. > > If the modules are installed, that's true. What if they are not? In that case you have to modify the load path, somehow. I have been doing this with scripts that add to $GUILE_LOAD_PATH and $GUILE_LOAD_COMPILED_PATH. (Which is another thing to note; if you have an autotooled project, the .go files go in the $builddir.) So if I'm hacking against an uninstalled tekuti, for example, I run my code within ~/src/tekuti/env. > For scripts that use uninstalled modules, then, some kind of solution is > needed; ideally one that works for both 1.8 and 1.9/2.0, allows the code > needed to live in a single common file, rather than duplicated at the > top of each script; and continues to work if the script+module tree as a > whole is moved to a different place in the filesystem. Also I think > it's preferable if the solution is a Guile one, as opposed to based on > the #! line, or needing a shell script wrapper. How would it look? I guess I am unclear on the actual problem being solved here :) Let's consider that I am hacking on my "analysis" script, which lives at ~/src/foo/analysis. I guess it's helping me in my "foo" project. Now the script gets too big; time to split into modules. How to do that? We have basically one option of a path to automatically add to the load path: ~/src/foo. It seems tractable, unless we start to consider installing the script to /usr/bin, combined with the presence of "" in the default load-extensions list, in which case the unexpected interactions with other members of that path look daunting. No, I think that the script itself will need to indicate some path to add to the load path. What you can do, perhaps, is add a macro: (define-syntax add-relative-load-path (lambda (x) (syntax-case x () ((_ path) (string? (syntax->datum #'path)) (let* ((src (syntax-source #'x)) (current-file (or (and src (assq-ref src 'filename)) (error "Could not determine current file name"))) (vicinity (dirname (canonicalize-path current-file))) (path-elt (in-vicinity vicinity (syntax->datum #'path #`(eval-when (compile load eval) (set! %load-path (cons #,path-elt %load-path Then in "analysis", you could `(add-relative-load-path ".")'. But... If you compile a .go, then install both .scm and .go somewhere, the installed .go file will reference the build path, which was inserted into the source via the current-file business. (You could argue that this is precisely the case that we are _not_ interested in, given that currently we only install .go files for files that are in the load path. But I guess we should figure out a better autocompilation story for files in $bindir.) It _is_ true that we need to figure out what's going on with (current-load-port) in 1.9, but it's not clear what to do, exactly, when you have compiled files; you could not have the corresponding .scm at all, or in any case when you've loaded the .go you don't actually have a port to the .scm, and in fact if the file was loaded via `load-from-path' you don't know exactly where the .scm is at all. Perhaps we should residualize into the .go whether the file was compiled for `load' or for `load-from-path', and what was the original path of the file. > Good point, thanks for the reminder about that. But (for 1.9/2.0) > `include' will always be well-defined and reliably relative to the > containing file's name, won't it? Yes, at expansion time. But note the .scm/.go installation case; and also note that the code that chooses when to use a .go over a .scm doesn't know anything about dependencies, currently, so a change to an included file doesn't trigger recompilation of the includer. Andy -- http://wingolog.org/
reminder: read NEWS
Hey all, I have noticed a number of questions coming across the list that almost seem to indicate that folks have not read the NEWS. I know this cannot be the case! ;) However if you're just jumping on the 2.0 train, take an hour or so out of your hacking to review the NEWS file. It is long, but should be exhaustive. If it's not, let us know. We'll be preparing a more condensed version for the release notes, but the long format is the most informative. Cheers, Andy -- http://wingolog.org/
Re: reminder: read NEWS
On 30 Jan 2011, at 13:20, Andy Wingo wrote: I have noticed a number of questions coming across the list that almost seem to indicate that folks have not read the NEWS. I know this cannot be the case! ;) However if you're just jumping on the 2.0 train, take an hour or so out of your hacking to review the NEWS file. It is long, but should be exhaustive. If it's not, let us know. We'll be preparing a more condensed version for the release notes, but the long format is the most informative. Since there are some significant changes between 1.8 and 2.0, perhaps the manual might have a special section listing them. Searching the 1.9 GIT manual for "1.8" indicates that this information is currently mainly given in a few footnotes. In the case Guile is used as a library (as in LilyPond), their users might consult the manual at need, but I suspect not the NEWS file.
