Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Chris Vine  writes:

> I have a pipeline macro which sort-of mimics ML's |> pipeline operator
> which I use a lot:
>
> (define-syntax ->
>   (lambda (x)
> (syntax-case x ()
>   [(k exp0 . exps)
>(let* ([reversed (reverse (cons (syntax->datum #'exp0)
>(syntax->datum #'exps)))]
>   [out (let loop ([first (car reversed)]
>   [rest (cdr reversed)])
>  (if (null? rest)
>  first
>  (let ([func (car first)]
>[args (cdr first)])
>(append `(,func ,@args)
>(list (loop (car rest) (cdrrest)))])
>  (datum->syntax #'k out))])))
>
> Because all the macro does is to rearrange input forms, this is hygienic
> without the need to manipulate syntax objects - you can convert to a datum,
> rearrange and then convert back to a syntax object again, as above.

This macro is *not* hygienic.  The calls to 'syntax->datum' strip all of
the context information from the syntax objects, and then build a new
expression using raw S-expressions.  The result is essentially the same
as if you used 'define-macro'.  This results various problems.

For example:

--8<---cut here---start->8---
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
(syntax-case x ()
  [(k exp0 . exps)
   (let* ([reversed (reverse (cons (syntax->datum #'exp0)
   (syntax->datum #'exps)))]
  [out (let loop ([first (car reversed)]
  [rest (cdr reversed)])
 (if (null? rest)
 first
 (let ([func (car first)]
   [args (cdr first)])
   (append `(,func ,@args)
   (list (loop (car rest) (cdr rest)))])
 (datum->syntax #'k out))])))
scheme@(guile-user)> (define t 'global-t)
scheme@(guile-user)> (define-syntax-rule (foo x)
   (-> x (format #t "[t=~A] ~A\n" t)))
scheme@(guile-user)> (let ((t 'inner-t)) (foo t))
[t=global-t] global-t
$1 = #t
scheme@(guile-user)> 
--8<---cut here---end--->8---

I recommend reformulating the -> macro using 'syntax-rules' as follows:

--8<---cut here---start->8---
scheme@(guile-user)> (define-syntax ->
   (syntax-rules ()
 ((-> exp)
  exp)
 ((-> exp ... (op args ...))
  (op args ... (-> exp ...)
scheme@(guile-user)> (let ((t 'inner-t)) (foo t))
[t=global-t] inner-t
$8 = #t
scheme@(guile-user)> 
--8<---cut here---end--->8---

This macro is hygienic, and also easier to comprehend (IMO).  Of course,
it could also be implemented using syntax-case.  The key is to always
work with the syntax objects.

Whenever you use 'syntax->datum' on expressions that are not purely
literals, you will be sacrificing hygiene.

  Regards,
Mark

Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Here's a complete, unedited transcript with Guile 2.2.6:

--8<---cut here---start->8---
mhw@jojen ~$ guile
GNU Guile 2.2.6
Copyright (C) 1995-2019 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
(syntax-case x ()
  [(k exp0 . exps)
   (let* ([reversed (reverse (cons (syntax->datum #'exp0)
   (syntax->datum #'exps)))]
  [out (let loop ([first (car reversed)]
  [rest (cdr reversed)])
 (if (null? rest)
 first
 (let ([func (car first)]
   [args (cdr first)])
   (append `(,func ,@args)
   (list (loop (car rest) (cdr rest)))])
 (datum->syntax #'k out))])))
scheme@(guile-user)> (define t 'global-t)
scheme@(guile-user)> (define-syntax-rule (foo x)
   (-> x (format #t "[t=~A] ~A\n" t)))
scheme@(guile-user)> (let ((t 'inner-t))
   (foo t))
[t=global-t] global-t
$1 = #t
scheme@(guile-user)> (define-syntax ->
   (syntax-rules ()
 ((-> exp)
  exp)
 ((-> exp ... (op args ...))
  (op args ... (-> exp ...)
scheme@(guile-user)> (let ((t 'inner-t))
   (foo t))
[t=global-t] inner-t
$2 = #t
scheme@(guile-user)>
--8<---cut here---end--->8---

Chris Vine  writes:
> How strange.  Both your and my macro gives 'global-t' when I test them,

Can you show me a complete, unedited transcript that demonstrates what
you're seeing?

> which is the result I would expect.  (Maybe I am missing something here,
> but a result of 'inner-t' would seem to me to imply unhygiene.)

(foo EXPR) is supposed to print "[t=global-t] VAL", where VAL is the
result of evaluating EXPR.  With this in mind,

(let ((t 'inner-t))
  (foo t))

The argument to 'foo' here should refer to the lexical binding of 't',
i.e. the variable with value 'inner-t'.  I'm curious what would make you
expect otherwise.

On the other hand, the reference to 't' in the template of the 'foo'
macro should refer to the toplevel variable 't', because the template
does not appear within the 'let'.

This is a good example of why syntax objects are needed, to distinguish
between these two references to distinct variables named 't'.  When you
convert the references to datums, the distinctions are lost.

 Regards,
   Mark

Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Chris Vine  writes:
> My version was hygienic when used in functions, but not when used as a
> macro in a macro.

The word "hygienic" should not be used to describe your original macro.
It would be more accurate to say that although your original version was
unhygienic, in practice it would work in many common cases.  That is
generally the case for unhygienic macros.

 Best,
  Mark

Re: Threading / Pipe Macro

[Mark H Weaver]

Hello again,

Chris Vine  writes:
> My version was hygienic when used in functions, but not when used as a
> macro in a macro.

Not to belabor the point, but I wanted to mention that the unhygienic
'->' macro may fail when used in procedures, even when '->' is never
used in the definition of another macro.  Here's an example, where 'cut'
and '->' are used in combination:

--8<---cut here---start->8---
scheme@(guile-user)> ,use (srfi srfi-26)
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
(syntax-case x ()
  [(k exp0 . exps)
   (let* ([reversed (reverse (cons (syntax->datum #'exp0)
   (syntax->datum #'exps)))]
  [out (let loop ([first (car reversed)]
  [rest (cdr reversed)])
 (if (null? rest)
 first
 (let ([func (car first)]
   [args (cdr first)])
   (append `(,func ,@args)
   (list (loop (car rest) (cdr rest)))])
 (datum->syntax #'k out))])))
scheme@(guile-user)> (define foo (cut -> <> (format #t "~A\n")))
;;; :17:12: warning: possibly unbound variable `t-15fc270a-2d'
scheme@(guile-user)> (foo 4)
:17:34: In procedure foo:
In procedure module-lookup: Unbound variable: t-15fc270a-2d

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> ,expand (cut -> <> (format #t "~A\n"))
$1 = (lambda (t-15fc270a-34-1)
  (format #t "~A\n" t-15fc270a-34))
scheme@(guile-user)> (define-syntax ->
   (syntax-rules ()
 ((-> exp)
  exp)
 ((-> exp ... (op args ...))
  (op args ... (-> exp ...)
scheme@(guile-user)> (define foo (cut -> <> (format #t "~A\n")))
scheme@(guile-user)> (foo 4)
4
$2 = #t
scheme@(guile-user)> ,expand (cut -> <> (format #t "~A\n"))
$3 = (lambda (t-15fc270a-59)
  (format #t "~A\n" t-15fc270a-59))
scheme@(guile-user)> 
--8<---cut here---end--->8---

So, more generally, unhygienic macros may cause problems when they are
used in combination with other macros.  Since macros are so ubiquitous
in Scheme, attempting to avoid such combinations is likely to be
brittle.

  Best,
   Mark

Re: Now crashing [was Re: guile-2.9.2 and threading

[Mark H Weaver]

Hi Linas,

> Investigating the crash with good-old printf's in libguile/vm.c produces
> a vast ocean of prints ... that should have not been printed, and/or should
> have been actual errors, but somehow were not handled by scm_error.
> Using today's git pull of master, here's the diff containing a printf:
>
> --- a/libguile/vm.c
> +++ b/libguile/vm.c
> @@ -1514,12 +1514,23 @@ thread->guard); fflush(stdout); assert (0); }
>
>proc = SCM_SMOB_DESCRIPTOR (proc).apply_trampoline;
>SCM_FRAME_LOCAL (vp->fp, 0) = proc;
>return SCM_PROGRAM_CODE (proc);
>  }
>
> +printf("duuude wrong type to apply!\n"
> +"proc=%lx\n"
> +"ip=%p\n"
> +"sp=%p\n"
> +"fp=%p\n"
> +"sp_min=%p\n"
> +"stack_lim=%p\n",
> +SCM_FRAME_SLOT(vp->fp, 0)->as_u64,
> +vp->ip, vp->sp, vp->fp, vp->sp_min_since_gc, vp->stack_limit);
> +fflush(stdout);
> +
>vp->ip = SCM_FRAME_VIRTUAL_RETURN_ADDRESS (vp->fp);
>
>scm_error (scm_arg_type_key, NULL, "Wrong type to apply: ~S",
>   scm_list_1 (proc), scm_list_1 (proc));
>  }
>
> As you can see, shortly after my printf, there should have been an
> error report.

Not necessarily.  Note that what 'scm_error' actually does is to raise
an exception.  What happens next depends on what exception handlers are
installed at the time of the error.

> There is no error report... until 5-10 minutes later, when the error
> report itself causes a crash.  Before then, I get an endless
> high-speed spew of prints:

It looks like another error is happening within the exception handler.

   Mark

PS: It would be good to pick either 'guile-devel' or 'guile-user' for
continuation of this thread.  I don't see a reason why it should be
sent to both lists.

Re: Write a macro which defines a procedure

[Mark H Weaver]

Matt Wette  writes:

> On 7/19/19 3:51 PM, zelphirkaltstahl wrote:
>> (module-define! (current-module)
>>  ;; `route` should be `/container/json` for example.
>>  route
>
> (quote route)

Yes, or equivalently:  'route

As an aside, is there a reason to not define it more simply as follows?

 Mark

--8<---cut here---start->8---
(define-syntax define-api-route
  (syntax-rules ()
[(define-api-route route http-method my-content-type)
 ;; `route` should be `/container/json` for example.
 (define* (route docker-socket #:key (data #f))
   (call-with-values
   (lambda ()
 ;; Here the route needs to be given as a string to `http-get`.
 (http-get (variable-name->string route)
   #:port docker-socket
   #:version '(1 . 1)
   #:keep-alive? #f
   ;; Why is my quasiquote not working as expected?
   #:headers `((host . ("localhost" . #f))
   (content-type . (,my-content-type (charset . 
"utf-8"
   #:body (scm->json-string data)
   #:decode-body? #t
   #:streaming? #f))
 (lambda (response response-text)
   (let ([resp-text-as-string (bytevector->string response-text 
"utf-8")])
 (cons response resp-text-as-string)]))
--8<---cut here---end--->8---

Re: Write a macro which defines a procedure

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl  writes:

> At first I had the macro use `define`, but then I thought: "What if I
> want to conditionally define a route procedure?". My guess is, that then
> the define form would be inside some `cond` or `if` form and then it
> would not work

You're right, it would not work.  Nonetheless, I would strongly
discourage you from doing this kind of thing unless there's a very
compelling reason for it.

Note that we have 'cond-expand' from SRFI-0 for conditional inclusion of
definitions, if you need it.  'cond-expand' is roughly analogous to #if
in C.  It's less general than plain 'if', but there's an important
reason for that, and it has to do with the distinction between compile
time and run time.

If you use 'cond-expand', the condition is checked at compile time, and
consequently it is known at compile time which branch was taken and
therefore which definitions were chosen.

If you use 'cond', then the condition will be checked at run time, which
will normally rule out the possibility of optimization opportunities
such as inlining and partial evaluation.  At present, Guile is not able
to perform these optimizations for toplevel level definitions in any
case, but it might be possible in the future, at least for modules that
are not explicitly mutated using low-level operations like this.

Also, it's possible that a future version of Guile may be more
disciplined in its handling of phases, i.e. expand-time
(i.e. compile-time) vs run-time phases.  See section 7.2 (Import and
export levels) of the R6RS for details on one approach, which would
bring many advantages.  It's been a while since I looked at this
closely, and at this point I would need some time to refresh my memory
of the details, but it seems to me that the code you've written would
likely cause problems with such an approach.

Both the R6RS and R7RS have more declarative module systems, which bring
many benefits.  Added freedom in one area often implies a loss of
freedom somewhere else.  In this case, the freedom to treat modules as
arbitrarily mutable objects implies a loss of freedom in the compiler to
make any assumptions about what's in the module or even its set of
bindings.  That freedom carries a very significant cost.

   Mark

Re: srfi-9 vs make-record-type

[Mark H Weaver]

Hi Christopher,

Christopher Lam  writes:

> Hi All
> In experiments converting legacy code to use srfi-9 records, I'm finding
> the latter doesn't travel well across modules.
>
> See simple code below -- m1.scm runs fine however m2.scm borks when
> creating srfi-9 record object
>
> Any clue why srfi-9 can't be exported?
>
> For various reasons I must keep (load "module.scm") mechanism
> m1.scm follows
> (use-modules (srfi srfi-9))
>
> (define-record-type 
>   (make-person name age)
>   person?
>   (name person-name set-person-name!)
>   (age person-age set-person-age!))
>
> (define  (make-record-type "pet" '(name age)))
> (define make-pet (record-constructor ))
> (define pet? (record-predicate ))
> (define pet-name (record-accessor  'name))
> (define pet-age (record-accessor  'age))
> (define set-pet-name! (record-modifier  'name))
> (define set-pet-age! (record-modifier  'age))
> (export make-person)
> (export make-pet)
>
> (display "pet ")
> (let ((pet2 (make-pet "milou" 7)))
>   (display (pet-name pet2)))
> (display ", person ")
> (let ((person2 (make-person "james" 54)))
>   (display (person-name person2)))
> (newline)
> m2.scm follows
> (load "m1.scm")
> (display "in m2:")
> (newline)
> (display "pet ")
> (let ((pet2 (make-pet "milou" 7)))
>   (display (pet-name pet2)))
> (display ", person ")
> (let ((person2 (make-person "james" 54)))
>   (display (person-name person2)))
> (newline)
> 
> $guile m1.scm runs successfully
>
> pet milou, person james
>
> $guile m2.scm first runs m1.scm but fails to recognise the srfi-9
> make-person exists:
> pet milou, person james
> in m2:
> pet milou, person Backtrace:
>6 (apply-smob/1 #)
> In ice-9/boot-9.scm:
> 705:2  5 (call-with-prompt _ _ #)
> In ice-9/eval.scm:
> 619:8  4 (_ #(#(#)))
> In ice-9/boot-9.scm:
>2312:4  3 (save-module-excursion _)
>   3831:12  2 (_)
> In /home/chris/sources/caca/m2.scm:
>  8:15  1 (_)
> In unknown file:
>0 (_ "james" 54)
>
> ERROR: Wrong type to apply: #

The problem here is that 'make-person' is a macro.  Macros are expanded
at compile time, and must therefore be available at compile time.

When you compile 'm2.scm', those macros are not available, because
'load' only has effects at run-time, by design.  Since there is no macro
definition for 'make-person' at compile-time, Guile's compiler generates
code that assumes it will be a procedure.

In summary, you cannot use 'load' to import macros from another module.

> For various reasons I must keep (load "module.scm") mechanism

If you can share those reasons, perhaps I can help you find another
solution.

  Regards,
Mark

Re: syntax-case identifier name predicate in guard expression

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl  writes:

> Hi Guile Users!
>
> I made some progress in writing a procedure defining macro for creating
> procedures which talk to an API.
>
> I now have working code, which checks the name of an identifier in a
> guard expression:
>
> 8<8<8<
> (use-modules (web uri)
>  (web client)
>  (json)
>  (ice-9 iconv)
>  (ice-9 regex))
>
>
> (define* (send-request-to-docker-socket request-url docker-socket
> my-content-type #:key (data #f))
>   (call-with-values
>   (lambda ()
>     (http-get request-url
>   #:port docker-socket
>   #:version '(1 . 1)
>   #:keep-alive? #f
>   #:headers `((host . ("localhost" . #f))
>   (content-type . (my-content-type (charset
> . "utf-8"
>   #:body (scm->json-string data)
>   #:decode-body? #t
>   #:streaming? #f))
>     (lambda (response response-text)
>   (let ([resp-text-as-string (bytevector->string response-text
> "utf-8")])
>     (cons response resp-text-as-string)
>
>
> (define-syntax define-api-route
>   (lambda (stx)
>     (define (identifier-name->string id)
>   (symbol->string (syntax->datum id)))
>
>     ;; Not needed yet.
>     ;; (define (identifier->symbol id)
>     ;;   (syntax->datum id))
>
>     (define (contains-url-template-variable? route-as-string)
>   (string-match "<[^>]+>" route-as-string))
>
>     ;; We do not need a macro to produce syntax. Instead we need to use
> a procedure to produce the
>     ;; syntax, because we want to use it while evaluating another macro.

Right.  Before, you were trying to perform the check within a procedural
macro, whose code was run when the 'define-api-route' macro was being
_defined_, which is too early.  Here you have the right idea.

>     (define make-simple-api-route-definition-syntax
>   (lambda (route http-method my-content-type route-as-string)
>     (syntax
>  (quote simple-route
>
>     (syntax-case stx (GET HEAD POST PUT DELETE CONNECT OPTIONS TRACE PATH)
>   [(_ route GET my-content-type)
>    (contains-url-template-variable? (identifier-name->string (syntax
> route)))
>    (syntax (quote aaa))]
>   ;; an else branch basically
>   [(_ route GET my-content-type) #t
>    (syntax
>     (define* (route docker-socket #:key (data #f))
>   (call-with-values
>   (lambda ()
>     ;; GET request because syntax GET was specified
>     (http-get request-url
>   #:port docker-socket
>   #:version '(1 . 1)
>   #:keep-alive? #f
>   #:headers `((host . ("localhost" . #f))
>   (content-type . (my-content-type
> (charset . "utf-8"
>   #:body (scm->json-string data)
>   #:decode-body? #t
>   #:streaming? #f))
>     (lambda (response response-text)
>   (let ([resp-text-as-string (bytevector->string
> response-text "utf-8")])
>     (cons response resp-text-as-string))])))
>
>
> (define* (connect-to-docker-socket #:key (socket-path
> "/var/run/docker.sock"))
>   (let ([docker-sock-addr (make-socket-address AF_UNIX socket-path)]
>     [docker-sock (socket PF_UNIX SOCK_STREAM 0)])
>     (setsockopt docker-sock SOL_SOCKET SO_REUSEADDR 1)
>     (connect docker-sock docker-sock-addr)
>     docker-sock))
>
>
> (display
>  (define-api-route /containers/json GET
> "application/x-www-form-urlencoded"))
>
>
> ;; (let ([dock-sock (connect-to-docker-socket)])
> ;;   (let ([resp-and-resp-text
> ;;  (/containers/json dock-sock
> ;;    #:data '(("all" . "true")
> ;; ("filters" . (("name" . #("db"))
> ;;   ("status" .
> #("running" "exited"))])
> ;; (display resp-and-resp-text)
> ;; (newline)))
>
>
> (display
>  (define-api-route /container/json GET "application/x-www-form-urlencoded"))
> 8<8<8<
>
> However, it seems, that now I have a new problem. It seems I cannot use
> a define form inside a syntax-case case!

No, that's not it.  The problem is not with the macro itself, but with
how you're attempting to use it.

The problem is simply that the macro expands into a definition, and you
are trying to use it within an expression context, where definitions are
not allowed, here:

> (display
>  (define-api-route /containers/json GET
> "application/x-www-form-urlencoded"))

This fails for the same reason that the following fails:

  (display (define foo 1))

> While it was possible to have a define form inside a syntax-rules,
> suddenly it seems impossible inside a syntax-case?

For purposes of this issue, it is irr

Re: yet another Guile tutorial for Linux (work in progress)

[Mark H Weaver]

Hi Basile,

Basile Starynkevitch  writes:

> I am writing yet another Guile tutorial. It is work in progress and
> available on http://starynkevitch.net/Basile/guile-tutorial-1.html
> under CC-BY-SA 3 license.
>
> Your comments are welcome.

Thanks for this.  I will try to find time to review it more carefully at
some point, but for now I have just one comment:

When referring to the operating system, please give GNU proper credit by
calling the system "GNU/Linux".  Linux is a kernel, not an operating
system.  The practice of calling the entire system "Linux" does great
harm to the GNU project, and arguably to the free software movement as a
whole, by greatly reducing the number of users of GNU/Linux systems who
learn about the crucially important freedoms that we are trying to
protect.

Anyway, I'm not sure why GNU/Linux is relevant to much (if any) of this
tutorial.  From a quick skim, I don't see anything here that wouldn't
work on FreeBSD, Solaris, or OS X.  Most of it would be the same on
Windows, for that matter.  The kernel is certainly not relevant.

Also, what do you mean by a "Linux developer"?  Do you mean to refer to
free software developers, or POSIX developers, or GNU/Linux developers,
or software developers in general?

It seems to me that most of the references to "Linux" are superfluous,
because almost all of this will work on any system that Guile supports.

What do you think?

  Regards,
Mark

Re: Problem installing the intended version

[Mark H Weaver]

Hi Thomas,

Thomas Morley  writes:

> recently I tested building LilyPond against guile-2.2.6 without
> success[*] and tried to investigate whether the problem is on the
> guile-side or at LilyPond (there are some patches which may or may not
> cause the problem).
>
> Though I stumbled across a problem which may be caused by guile or my
> naivity ...
>
> I had built guile-2.2.6 from the git repository and installed it via
> `sudo make install´
> Then I did (in/for a fresh repo):
>
>   git clone git://git.sv.gnu.org/guile.git ~/guile-2.2.4
>   cd guile-2.2.4/
>   git checkout v2.2.4
>   git checkout -b guile-2.2.4
>   sh autogen.sh
>   ./configure
>   make
>
> All successfull, then some tests:
>
>   ./meta/guile --version
>   -->  guile (GNU Guile) 2.2.4 [...]
>   ./meta/guile-config --version
>   --> guile-config - Guile version 2.2.4
>
> Then installed it systemwide
>
>   sudo make install
>
> And made some testings:
>
>   which guile
>   --> /usr/local/bin/guile
>   /usr/local/bin/guile --version
>   --> guile (GNU Guile) 2.2.6.1-a69b5
>
> Obviously 2.2.4 was _not_ installed but 2.2.6 from another repo.
>
> What am I missing?

What's happening here is that although the executable
/usr/local/bin/guile is from 2.2.4, it is being dynamically linked to
the newer libguile-2.2.so.1.4.1 from 2.2.6 that you installed earlier.
The libguile from guile 2.2.4 is libguile-2.2.so.1.3.1.

This happens because you never uninstalled the newer
libguile-2.2.so.1.4.1, and the library versioning information that we
assigned indicates that libguile-2.2.so.1.4.1 is a drop-in replacement
for libguile-2.2.so.1.3.1.  In other words, to our knowledge, any
executable linked with libguile-2.2.so.1.3.1 should also work properly
when linked with libguile-2.2.so.1.4.1.  That's because we added new
interfaces but did not change or remove any existing interfaces.

If you want to prevent libguile-2.2.so.1.4.1 from guile 2.2.6 from being
used, you should delete it, and then run 'ldconfig' as root to update
the links and caches.  Hopefully after that, libguile-2.2.so.1 and
libguile-2.2.so should be symlinks to libguile-2.2.so.1.3.1.

A more thorough approach, to make sure that no components of 2.2.6 are
being used, would be to run "make uninstall" from the Guile 2.2.6 build
directory (if you still have it), then rerun "make install" from the
Guile 2.2.4 build directory, and finally "ldconfig" as root.

   Best,
Mark

Re: Problem installing the intended version

[Mark H Weaver]

Hi Thomas,

Thomas Morley  writes:

> In the light of your explanations I think the current problem only
> occurs for _equal_ major/minor with _different_ micro versions.
> Correct?

Yes.

>> and finally "ldconfig" as root.
>
> Doing so I get:
> /sbin/ldconfig.real: /usr/local/lib/libguile-2.0.so.22.8.1-gdb.scm is
> not an ELF file - it has the wrong magic bytes at the start.
> /sbin/ldconfig.real: /usr/local/lib/libguile-2.2.so.1.3.1-gdb.scm is
> not an ELF file - it has the wrong magic bytes at the start.
>
> And I've got even more notifications of this kind before I started to
> do the "make uninstall"-thingy.
> Is this expected?

Yes.  The warnings are harmless, although mildly annoying.

The issue is that GDB wants us to put that .scm file in the same
directory as libguile, in order to have GDB auto-load Guile debugging
support for executables linked with libguile.  See section 23.4.1 (The
'OBJFILE-gdb.EXT' file) in "Debugging with GDB":

  
https://sourceware.org/gdb/current/onlinedocs/gdb/objfile_002dgdbdotext-file.html

Unfortunately, 'ldconfig' warns about it.  I seem to recall that we
reported the problem to the GDB developers at some point in the past.
Nowadays, most of the Guile developers seem to be using GNU Guix, where
'ldconfig' is never run, so we tend to forget about it.

 Regards,
   Mark

Re: How to use-modules within macro?

[Mark H Weaver]

Hi Florian,

"pelzflorian (Florian Pelz)"  writes:

> I am writing a Guile macro to manipulate Scheme code and am stuck on
> what I hope is a simple problem and it would be nice if you could
> explain.  I try:
>
> (define-syntax O
>   (lambda (x)
> (syntax-case x ()
>   ((_)
>#`(begin (use-modules (ice-9 local-eval))
> (local-eval 42 (the-environment)))
> (pk (O))

This approach is misguided and unnecessary.  You don't need to include
'use-modules' in your macro expansion, and it's best avoided.

The references to 'local-eval' and 'the-environment' in the macro
template above will refer to bindings present in the module where 'O' is
defined, *not* the module where 'O' is used.  This is part of what it
means to say that 'O' is a "hygienic" macro.

Therefore, all you need to do is make sure (ice-9 local-eval) is
imported in the module where 'O' is defined, like this:

--8<---cut here---start->8---
(define-module (my-module-that-exports-O)
  #:use-module (ice-9 local-eval)
  #:export (O))

(define-syntax O
  (lambda (x)
(syntax-case x ()
  ((_)
   #`(local-eval 42 (the-environment))
--8<---cut here---end--->8---

Does that work for you?

FYI, the way this works internally is that the macro expander operates
on "syntax objects" instead of plain S-expressions.  The main difference
is that "syntax objects" keep additional information about the lexical
environments where the embedded identifiers were originally found.  So
when a use of (O) expands into (local-eval 42 (the-environment)), the
identifiers 'local-eval' and 'the-environment' are looked up in the
proper environment.

By the way, another consequence of hygiene, which you probably don't
want here, is that the (the-environment) above will capture the lexical
environment where 'O' was *defined*, instead of the environment where
(O) is used.  In other words, in (let ((x 5)) (O)), the captured lexical
environment will not include 'x'.

I should also mention that using (the-environment) will pretty much
disable most compiler optimizations that would otherwise occur with that
top-level form.  That entire mechanism is best avoided if at all
possible.

Can you tell me more broadly what you are trying to accomplish here?
I may be able to suggest an alternate approach.

 Best,
  Mark

Re: How to use-modules within macro?

[Mark H Weaver]

Hi Florian,

"pelzflorian (Florian Pelz)"  writes:

> To retain unhygienic references, I am now using datum->syntax instead
> of local-eval.  It is much better.  For example, to make a macro that
> increments all numbers in a given program by one:
>
> (use-modules (ice-9 match))
> (define-syntax one-more
>   (lambda (x)
> (syntax-case x ()
>   ((_ exp)
>(datum->syntax
> #'exp
> (let loop ((y (syntax->datum #'exp)))
>   (match y
> ((? number?) (1+ y))
> ((? list?) (map loop y))
> (else y
>
> (let ((four 4))
>   (one-more (* 2 3 four)))
>
> Yields 48.  I hope this is the right approach for rewriting programs.

There are some problems above:

(1) The first argument to 'datum->syntax' must be an identifier, which
is the syntax object corresponding to a symbol.  Here, you are
passing an entire expression, and in the example usage above, #'exp
will be the syntax object corresponding to (* 2 3 hour).  Guile
should ideally raise an error in this case.

(2) The way you are doing things here destroys hygiene within the
expression that you are rewriting.  You convert the entire
expression with 'syntax->datum', process the datum, and then convert
the rewritten expression using 'datum->syntax'.  The problem here is
that 'syntax->datum' discards all of the extra information about
lexical environments of identifiers that were kept in the syntax
object.  This will cause severe problems when 'one-more' is used in
combination with other macros, including unintended variable
capture.

To do this properly, you must do the rewriting on the syntax objects
themselves.  It's okay to convert a syntax object to a datum to test
whether it's a literal number, but the important thing is that all
*identifiers* in the rewritten code should be preserved.

So, instead of using 'match' on the result of 'syntax->datum', you
should instead use 'syntax-case' on the syntax object itself, like this
(untested):

  (let loop ((e #'exp))
(syntax-case e ()
  (num
   (number? (syntax->datum #'num))
   #'(1+ num))
  ((x ...)
   (map loop #'(x ...)))
  (y
   #'y)))

Finally, I should mention that macro expansion is always done from the
outside in, meaning that when 'one-more' is expanded, its operand will
not yet have been expanded.  In general, this means that it's impossible
to comprehend the code within a macro's operands unless the parsing code
knows about every macro that might be used within the operands.  It's
not even possible to know which subparts are expressions and which are
other things like variable binding lists.

For this reason, I think it's generally a mistake to try to parse code
within a macro's operands.  It normally only makes sense for macros to
inspect the parts of operands that are considered part of the macro's
syntax.  For example, it makes sense for a 'let' macro to parse its
binding list, or for a 'match' macro to parse its patterns and
templates, but it does *not* make sense for a macro to try to parse
general subexpressions passed to the macro.

If you could give me a birds-eye view of what you're trying to do here,
I might be able to suggest other approaches.

   Best,
Mark

Re: extract documentation from (define ...

[Mark H Weaver]

Hi Jesse,

Jesse Gibbons  writes:

> I am trying to generate documentation for a project written in guile. I
> tried "guild doc-snarf" [0] but apparently it only recognizes
> documentation in double-semicolon comments. A lot of the project is
> documented in a string like one would document an emacs-lisp function.
> Is there a program to extract that kind of documentation?
>
> [0]: http://git.savannah.gnu.org/cgit/guile.git/tree/module/scripts/doc
> -snarf.scm

I would use 'object-documentation' procedure from (ice-9 documentation),
which is what underlies the ,describe REPL command.

   Best,
Mark

Re: How to use-modules within macro?

[Mark H Weaver]

Hello again,

I wrote earlier:
> So, instead of using 'match' on the result of 'syntax->datum', you
> should instead use 'syntax-case' on the syntax object itself, like this
> (untested):
>
>   (let loop ((e #'exp))
> (syntax-case e ()
>   (num
>(number? (syntax->datum #'num))
>#'(1+ num))
>   ((x ...)
>(map loop #'(x ...)))
>   (y
>#'y)))

I should mention that the use of 'map' directly on a syntax object is
only allowable in certain cases.  Here, we assume that the syntax object
produced by #'(x ...) is a normal Scheme list.  That is _not_ the case
for arbitrary syntax objects that correspond to a list.  For example, it
would _not_ be safe to pass 'e' as the list argument to 'map', although
'e' is in some sense equivalent to #(x ...) in the second clause above.

The reason is that syntax objects contain information about the
associated lexical environment which starts at the top of the expression
tree, and is *lazily* pushed down into the subexpressions as the tree is
taken apart using 'syntax-case' and put back together using 'syntax'
(a.k.a. "#'").

As a result, there are only a few cases when you can safely assume that
the top structure of a syntax object is a normal list or pair, and they
are spelled out in the documentation for 'syntax' in the R6RS Standard
Libraries specification:

  http://www.r6rs.org/final/html/r6rs-lib/r6rs-lib-Z-H-13.html#node_sec_12.4

Here's the relevant excerpt:

  The output produced by syntax is wrapped or unwrapped according to the
  following rules.

  * the copy of ( . ) is a pair if  or  contain any
pattern variables,

  * the copy of ( ) is a list if  contains any pattern
variables,

  * the copy of #( ... ) is a vector if any of , ..., 
contain any pattern variables, and

  * the copy of any portion of  not containing any pattern variables
is a wrapped syntax object.

A "wrapped syntax object" is one where the lexical environment
information has not yet been pushed down into the subexpressions.  It is
a special kind of object that you can only take apart using
'syntax-case'.

So, in the clause above where 'map' is used, 'e' might be a "wrapped
syntax object", but when the elements are extracted from it using the
'syntax-case' pattern (x ...) and then put back together using
#'(x ...), you can then assume that the resulting syntax object is
a normal Scheme list of syntax objects, and therefore it is safe to
use 'map' on it.

   Mark

Stepping back up as a co-maintainer

[Mark H Weaver]

Hello all,

In light of recent events, I've decided to step back up as a
co-maintainer of GNU Guile.

 Thanks,
   Mark

Re: Weird Guile Scheme Behaviour

[Mark H Weaver]

Hi Philip,

phi...@warpmail.net (Philip K.) writes:

> I was reading a thread on an imageboard[0] the other day, then I came
> across a most peculiar "bug", if it even is one. Since the original
> example was a bit dense (it tried to solve a problem someone else had
> posted, that's not relevant here), I tried to construct a minimal
> working example to discuss here.
>
> Compare
>
> (define (reverse-iota-1 max)
>   (let ((numbers '(start)))
> (let loop ((val 0))
>   (append! numbers
>(if (< max val)
>'()
>(begin
>  (loop (1+ val))
>  (list val)
> numbers))
>
> and
>
> (define (reverse-iota-2 max)
>   (let ((numbers '(start)))
> (let loop ((val 0))
>   (append! numbers
>(if (< max val)
>'()
>(begin
>  (loop (1+ val))
>  (list val)
> (cdr numbers)))
>
> (I know, the style is horrible, but that's not the point. Also, both
> have an internal state, so you have to re-eval the function every time
> before starting the function itself.)
>
> The only difference is in the last line. The first function returns the
> entire list (with the start symbol), and the second tries to chop it
> off.
>
> But what happens is that (reverse-iota-1 4) evals to '(start 3 2 1 0)
> while (reverse-iota-2 4) just returns '()!

The problem here is that '(start) is a *literal* list, and it's an error
to modify literals in Scheme.  In other words, modifying a literal
results in undefined behavior.  Like C string literals, Scheme literals
are actually part of the program text itself.

To fix this problem, instead of initializing 'numbers' to point to a
literal list, you must construct a fresh mutable list.  One way is by
replacing '(start) with (list 'start).

What's happening here is this: Since it's an error to modify a literal
list, and since the 'numbers' variable is never 'set!' within its
lexical scope (the only scope where a variable can possibly be set! in
Scheme), Guile's compiler is allowed to assume that 'numbers' will
always point to the literal '(start), and therefore that (cdr numbers)
can be optimized to '().

The other problem, as Neil and Vladimir correctly pointed out, is that
'append!' is permitted, but not required, to modify the original list.
Therefore, you must always do (set! numbers (append! numbers ...)) to
avoid relying on unspecified behavior.

Note that it's impossible in Scheme to implement an 'append!' procedure
that always modifies the original list.  Specifically, an empty list
cannot be destructively modified.  It can't be done for the same reason
that you cannot implement a C function with the following specification:

  struct pair { void *item; struct pair *next; };
  void appendx (struct pair *list_to_modify, struct pair *list_to_add);

This C function can only be implemented in the case where
'list_to_modify' has at least one element.  In that case, the 'next'
field of the last pair can be modified.  If 'list_to_modify' is NULL, it
can't be done, because 'appendx' doesn't have access to the variable
that contained NULL.

In Scheme, the same issues apply.  If there's at least one element in
the first list passed to 'append!', it can use 'set-cdr!' to modify the
last pair of the list.  If the first argument is '(), it can't be done.

In practice, that's the reason why 'append!' is specified the way it is.
However, I would advise against assuming that 'append!' will always
modify the original list if it's nonempty, because that fact, although
true of the current implementation, is not actually specified.

  Regards,
Mark

Re: Stepping back up as a co-maintainer

[Mark H Weaver]

zx spectrumgomas  writes:
> It's milk
> https://lists.gnu.org/archive/html/guile-devel/2019-10/msg00031.html

I'm sorry, but I don't understand what this means.  If you are accusing
me of something, can you state your case clearly, without reference to
proverbs?

   Mark

Re: Stepping back up as a co-maintainer

[Mark H Weaver]

zx spectrumgomas  writes:

> Of your words: “RMS has not yet appointed me as a co-maintainer.” , the
> word “yet” I guess that means if he would have had his permission then you
> wouldn't have needed it of the current maintainers, as I also deduce from
> here:
> “Frankly, it should not be your decision to make.”
> https://lists.gnu.org/archive/html/guile-devel/2019-10/msg00021.html

Yes, and I stand by that statement.  I made my case clearly in the
message cited above.

That said, I agree that it would have been improper for RMS to appoint
me without first consulting the existing maintainers.  In fact,
discussions are currently ongoing with Andy and Ludovic to resolve the
situation, and I'm hopeful that we'll reach a mutually acceptable
resolution.

Regarding my phrase "In light of recent events", which I cited as the
reason for my decision to return, I should now be clear what I meant by
that:

GNU is far more fragile than I had supposed.  It's like if your mother
has a heart attack, and you suddenly realize that although you've always
taken her presence in your life for granted, her mortality has now
become real to you in a way that it wasn't before.

I realize now that I took GNU's continued existence and integrity for
granted, like a steady rock that I naively assumed would be there and
stay strong regardless of whether I was a part of it or not.  Now I see
that it will depend on all of us who care about software freedom to do
what we can to keep GNU strong.

I'm not interested in dividing the Guile community.  That would
obviously be disastrous.  I fully support Andy's work, and I recognize
that he is the main driver in core Guile development.  His continued
participation is crucial.  Still, we must guard against Guile becoming a
one-man show if it can possibly be avoided.  Keeping GNU strong means,
among other things, resolving our differences peacefully and working
together to make Guile better.  That's what I intend to do.

I hope that clarifies things.

  Thanks,
Mark

Re: Interactive Debugging

[Mark H Weaver]

"Thompson, David"  writes:

> On Fri, Oct 18, 2019 at 8:40 AM Matt Wette  wrote:
>>
>> On 10/17/19 9:39 PM, Christopher Howard wrote:
>> > Hi, it seems like with the flexibility of Guile, I should be able to do
>> > something like this:
>> > ```(define (foo)  (let ((a 1)(b 2)(c 3))(jump-into-
>> > debugging-repl)))```
>> > And have access to a, b, and c and their values. But I'm not quite
>> > figuring out how to this.
>> >
>>
>> Define dump-into-debugging-repl as
>>
>> (start-repl #:debug (make-debug (stack->vector (make-stack #t)) 0
>> "trap!" #t))
>
> I think something like this should be shipped in the (system repl
> debug) module.  I'm used to quick and easy access to a debugger in
> other languages, such as Ruby, and having to wrap your head around the
> REPL, debug, and stack code is too much to ask of any new Guiler.

For what it's worth, I think something like this would be a welcome
addition.  However, my knowledge of the Guile debugger is currently
quite weak, so I'd prefer to hear from Andy before proceeding.

 Thanks,
   Mark

Re: Diversification [ branched from Re: conflicts in the gnu project now affect guile]

[Mark H Weaver]

Hi Todor,

Todor Kondić  writes:

> [...]  I've set up my workflows around Guix, git(lab)
> and a customised Emacs installation (instead of R Studio). My small
> team of science students (majority female, various cultural
> backgrounds), never previously exposed to a GNU system to such an
> extent, managed to get a handle on it quite impressively.
>
> But, I doubt any of them would find it natural to take a step further
> and participate in GNU itself (ugh, now I sound like a preacher of a
> new age religion). To my knowledge, interaction within GNU communities
> is still mostly mailing lists and IRC. This _not_ my students' natural
> digital habitat. I am probably not saying anything new, though ...

You raise an important issue.  If we can improve the situation without
causing other problems, I think we should.  I don't know of any modern
replacement for mailing lists that has the properties we need, but I
*do* think there's a very promising alternative for live chat: Matrix.
Amirouche mentioned it elsewhere in this thread.

  https://matrix.org/

Matrix is supported by a very large and diverse set of free clients,
from modern Web-based interfaces to simple text programs, multiple
Emacs-based clients, and several gateways to other protocols such as
IRC, so that old-timers can use their preferred IRC client if they
prefer.

  https://matrix.org/clients/

Incidentally, there was recently an internal GNU project discussion
about how to better communicate with one another, and Matrix was
identified as an option that would meet our requirements.

The client that would likely be most attractive for the younger
generation is Riot.im:

  https://about.riot.im/

What do you think?

Thanks,
  Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

John Cowan  writes:

> +1.  If only it weren't GPL3, which makes it ineligible to be a SRFI
> implementation

Zelphir could propose a SRFI for the API only, with no reference
implementation.  The existing implementation could remain copylefted.

   Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

I wrote:
> Zelphir could propose a SRFI for the API only, with no reference
> implementation.  The existing implementation could remain copylefted.

John Cowan  replied:
> Unfortunately not.  A SRFI must have a sample implementation to get
> finalized (otherwise it is feared there would be too many WIBNI SRFIs).
> The implementation doesn't have to be portable, but it has to have the
> correct license.

That's unfortunate.  I stand corrected.

Nonetheless, I find it regrettable that you've chosen to pressure a
library author on a GNU project forum to abandon copyleft licensing.
Personally, I don't think that SRFIs are important enough to justify
bowing to such pressure.  Guile-JSON's association with Guile and Guix
is enough for it to gain wide adoption, whether it is a SRFI or not.
That said, it is of course Aleix's decision how to license his library.

  Thanks,
Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

Hi Aleix,

Aleix Conchillo Flaqué  wrote:
> If at some point I decide to switch to the MIT license I assume
> the projects that currently use guile-json would still be able to use it
> (probably renamed to some SRFI number) the same way they use other SRFIs,
> right?

Yes, as long as it's GPL-compatible.  The X11 and Expat licenses are
both GPL-compatible.  The ambiguous term "MIT license" usually refers to
one of those two licenses.

  Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

Hi John,

John Cowan  writes:

> Distinguo.  I have not *pressured* the author to dual license; that is, I
> have not exercised force or undue influence (of which I have none).

According to , "pressure"
means: "To encourage or heavily exert force or influence", and according
to , "encourage" means
(among other senses) "To spur on, strongly recommend."

As an example use of the word "pressure", wiktionary offers this:

  Do not let anyone pressure you into buying something you do not want.

For another source, according to
:

  "If you put pressure on someone, you try to cause that person to do
  something by persuading or threatening them"

Here, I used the word "pressure" in the sense of to "persuade" or to
"spur on, strongly recommend."

Therefore, I think I used the word correctly here.

  Thanks,
Mark

Re: guile-user Digest, Vol 204, Issue 2

[Mark H Weaver]

John Cowan  wrote:
> Dual licensing is also a possibility, so that users who prefer the GPLv3
> license would still have it.

Dual licensing with a lax license option would eliminate the main
advantage of copyleft, namely that it offers an inducement to those who
wish to use Guile-JSON to release their code as free software.

See  for more.

 Thanks,
   Mark

Re: guile-json, SRIFs and licenses

[Mark H Weaver]

Hi John,

John Cowan  writes:

> [...]  There are two traditional arguments
> in favor of putting libraries under the GPL:

[...]

> 2) "Benefit GPLed programs":  if a library is GPLed, it supposedly gives
> the advantages of using that library only to GPLed applications.

You're mistaken.  A GPLed library can be used by any program covered by
a *GPL-compatible* license.

> I think history shows that this doesn't work very well in the long run:
> rather than accepting the GPL, a lot of duplicative effort was put into
> libedit/editline,

It's a mistake to conclude from this that it didn't work well.  Avoiding
duplicative effort is _not_ a core goal of the Free Software Movement.

A vastly more important goal is that free software should exist for all
of the jobs that people want to use their computers for, so that users
need not sacrifice their freedom to do their computing.  The GNU GPL has
helped *enormously* with that goal.  For example, we gained our first
free C++ compiler thanks to the GNU GPL, because they built it on GCC.
Ditto for Objective C, and CLISP.

Those are major historical examples, but the same thing happens quite
frequently and unremarked in smaller examples, whenever a boss would
reflexively tell an employee that a program they wrote should be kept
proprietary, but is compelled to make it free software because it
depends on a copylefted library (such a Guile-JSON).

See  for more on this.

Providing an inducement to others make more software into free software
is in fact the main goal of the GNU GPL, and you didn't even include it
in your list of "traditional arguments in favor of putting libraries
under the GPL".

> Whether or not a GPLed JSON library requires the Scheme implementation
> to be itself GPL depends on the implementation, but certainly a
> stand-alone *application* that uses it would have to be.

Again, you are mistaken.  Check your facts, please.  See
.

  Thanks,
Mark

Re: Weird behavior of hash-table

[Mark H Weaver]

Hi Zelphir,

Zelphir Kaltstahl  writes:

> I've noticed a strange behavior of hash tables. I put in symbols as keys
> and integers as values, but when I try to get the integers out again by
> using the same symbol, I get back a #f instead. Here is the code I am using:
>
> 
> (use-modules
>  ;; SRFI 60: procedures for treating integers as bits
>  (srfi srfi-60)
>  (ice-9 hash-table))
>
>
> (define SQUARES
>   ;; vector, constant time access
>   #('A1 'B1 'C1 'D1 'E1 'F1 'G1 'H1
> 'A2 'B2 'C2 'D2 'E2 'F2 'G2 'H2
> 'A3 'B3 'C3 'D3 'E3 'F3 'G3 'H3
> 'A4 'B4 'C4 'D4 'E4 'F4 'G4 'H4
> 'A5 'B5 'C5 'D5 'E5 'F5 'G5 'H5
> 'A6 'B6 'C6 'D6 'E6 'F6 'G6 'H6
> 'A7 'B7 'C7 'D7 'E7 'F7 'G7 'H7
> 'A8 'B8 'C8 'D8 'E8 'F8 'G8 'H8))

I guess that you meant for this to be a vector of symbols.  In fact, it
is a vector of lists of the form (quote ), for which '
is a shorthand.

  scheme@(guile-user)> (vector-ref SQUARES 0)
  $10 = (quote A1)

To fix the problem, remove all of the apostrophes (') from the elements
of the vector literal above.  Like list literals, vector literals take
raw values, not expressions.

> (hash-ref SQUARES-TO-INTEGERS 'A1)

  scheme@(guile-user)> (hash-ref SQUARES-TO-INTEGERS 'A1)
  $11 = #f
  scheme@(guile-user)> (hash-ref SQUARES-TO-INTEGERS ''A1)
  $12 = 1

   Mark

Re: Weird behavior of hash-table

[Mark H Weaver]

Hi tomás,

 wrote:

> So the best thing for one's brain is to train it to read #(...)
> as some weird relative of '(...)?

Yes.  They are both literals, and no part of a literal is evaluated.
#(...) is actually a shorthand for '#(...), and incidentally, one which
was not standardized until the R7RS.  In portable R[3456]RS code, you
must explicitly quote vector literals.

It's important to distinguish between an expression and the value(s) it
evaluates to.  It's easy to confuse them in Scheme, where every
expression is a value, but not necessarily the same value that it
evaluates to.  For example, the expression (+ 1 1) is a list of length 3
which evaluates to the number 2.  Similarly, 'A1, which is a shorthand
for (quote A1), is a list of two symbols which evaluates to the symbol
A1.  Only the "self-evaluating" values evaluate to themselves.

In general, you should not put a quote (') anywhere that you could not
write (+ 1 1) in place of 2, unless of course you *want* a list starting
with the symbol 'quote'.

> Is there a corresponding weird relative of `(...)?

Yes.  Remember that #(...) is a shorthand for '#(...).  You can replace
the "'" with "`", just as you would for a quasiquoted list.  For
example:

  scheme@(guile-user)> `#(1 2 3)
  $1 = #(1 2 3)
  scheme@(guile-user)> `#(1 ,(+ 1 1) 3)
  $2 = #(1 2 3)
  scheme@(guile-user)> `#(1 (+ 1 1) 3)
  $3 = #(1 (+ 1 1) 3)

  Regards,
Mark

Re: UTF16 encoding adds BOM everywhere?

[Mark H Weaver]

Hi,

Jean Abou Samra  wrote:

> With this code:
> 
> (let ((p (open-output-file "x.txt")))
>    (set-port-encoding! p "UTF16")
>    (display "ABC" p)
>    (close-port p))
> 
> the sequence of bytes in the output file x.txt is
> 
> ['FF', 'FE', '41', '0', 'FF', 'FE', '42', '0', 'FF', 'FE', '43', '0']
> 
> FFE is a little-endian Byte Order Mark (BOM), fine.
> But why is Guile adding it before every character
> instead of just at the beginning of the string?
> Is that expected?

No, this is certainly a bug.  It sounds like the
'at_stream_start_for_bom_write' port flag is not being cleared, as it
should be, after the first character is written.  I suspect that it
worked correctly when I first implemented proper BOM handling in 2013
(commit cdd3d6c9f423d5b95f05193fe3c27d50b56957e9), but the ports code
has seen some major reworking since then.  I guess that BOM handling was
broken somewhere along the way.

I would suggest filing a bug report.  I don't have time to look into it,
sorry.  I don't work on Guile anymore.  I only happened to see your
message by chance.

 Regards,
   Mark

-- 
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about .

Re: Ignore errors while evaluating with `primitive-eval'

[Mark H Weaver]

Hi,

Alexandru Cojocaru  writes:

> is it possible to ignore errors when evaluating a badly formed
> expression?

Yes.  An easy way is to use 'false-is-exception', like this:

  (false-if-exception (primitive-eval '(this is an error)))
  => #f

However, this won't work if you need to tell the difference between the
evaluation returning #f and an error happening.  In that case, try this:

  (catch #t
(lambda ()
  (primitive-eval '(this is an error)))
(lambda (key . args)
  (display "An error occurred during evaluation.\n")))

  Happy hacking!
  Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Panicz Maciej Godek  writes:

> 2013/8/23 Eli Zaretskii 
>
> 
> > I managed to go through the compilation process,
> > and (after modifying meta/Makefile.am according to Eli's
> > patch) to make install.
> 
> 
> This compilation process includes compiling all the Scheme files
> that
> come with Guile. Since you say it succeeded, I don't understand
> what
> you say later, see below. Do you see a lot of *.go files in the
> directory where you built Guile?
>
> When I enter the build directory and type
> $ find ./ -name '*.go'
> it seems that every .scm file from `modules' directory has its .go
> counterpart,

Note that setting GUILE_LOAD_PATH is not enough.  You must also set
GUILE_LOAD_COMPILED_PATH to include the top directory where the .go
files are installed.  Specifically, both of these should point to a
directory that contains "ice-9", so that
$GUILE_LOAD_PATH/ice-9/boot-9.scm exists, and so does
$GUILE_LOAD_COMPILED_PATH/ice-9/boot-9.go.

A better solution is to provide a proper --prefix, so that the files are
in the default load paths in the built Guile.

Of course, ideally we would allow the entire Guile install to be freely
relocatable, as Windows users have come to expect.  This would require
some help from a Windows developer.

Someone on IRC (who prefers to stay anonymous) once outlined this plan
to me:

  infer paths in the usual way - by saving the hInstance handle that it
  gets in DllMain(), then using GetModuleFileName () on that handle to
  obtain location of the libguile-whatever.dll, then checking if that dll
  is in a "bin" directory, and taking parent directory of that "bin"
  directory as the prefix, and then appending things like "/etc/config.d"
  and "/share/guile/stuff" to that prefix

  Well, an easy way to fix this is to separate the fix into two parts. One
  part is about getting the prefix at runtime (on POSIX you can just use a
  mockup that gets it from some envvar; later you'll be able to add a
  W32-only implementation that does what i said earlier), and the other
  makes guile use that prefix to construct paths it needs

If there are no objections to this approach, I'd be willing to work with
a Windows developer to make it happen.

Regards,
  Mark

Re: [ANN] Guile-WM 0.1/Guile XCB 1.1

[Mark H Weaver]

Hi Mark,

m...@markwitmer.com writes:

> I feel like it's time to announce Guile-WM, my stab at an all-Guile
> Window Manager. It's actually more of a toolkit for writing X
> applications, with the window manager stuff being what's implemented so
> far. It includes live REPL support, the usual
> keymap/minibuffer/user-defined command paradigm, and more. [...]
>
> I've also dramatically revamped Guile XCB.

This all sounds very exciting!  Enough so that I finally pulled
guile-xcb and attempted to build it from git.  Unfortunately, I ran into
several problems:

* Although the README says that Guile 2.0.9 is required, the configure
  script will accept nothing less than Guile 2.2.  Since 2.2 does not
  exist yet, it would be good to accept Guile 2.0 in the meantime :)

* After changing 2.2 -> 2.0, the next problem is that the build process
  seems to assume that (xcb xml) is already in Guile's %load-path, but
  the build system doesn't arrange for this.  I worked around this by
  setting GUILE_LOAD_PATH to the build directory.

* After setting GUILE_LOAD_PATH to the build dir, it then tries to load
  the (xcb xml xproto) module, but xproto.scm does not exist anywhere in
  the source tree.

At this point, I give up.  Can you help?

 Thanks,
   Mark

Re: asynchronous socket library

[Mark H Weaver]

Hi Mark,

Sorry for taking so long to respond.

m...@markwitmer.com writes:

> Mark H Weaver  writes:
>
>> m...@markwitmer.com writes:
>>> Reading and writing to a socket seems to lend itself well to custom
>>> binary ports,
>>
>> Why do you say that?  For most purposes, Guile's native file ports are
>> superior, and they seem the natural choice for sockets.
>>
>
> I over-generalized. In my application, I'm using the X protocol, which
> is a binary one. I've been so involved with that I forgot that much of
> the time you want to read/write character data from a socket.

Guile's native file ports (which also work on sockets) can be used for
binary data as well as character data.  There's no need to create a
custom binary port for this purpose.  Indeed, the custom ports have
several major disadvantages compared to the native file ports.

>> 'get-bytevector-some' from (rnrs io ports) might do what you need.  If
>> there's at least one byte available it will not block, but in practice
>> it reads much more than that (for buffered ports).  Guile 2.0.9 has a
>> nice fast implementation.  Prior to 2.0.9, it was less so.
>>
>> In more detail: as of 2.0.9, 'get-bytevector-some' will simply copy the
>> input buffer managed by Guile into a bytevector and return it.  If
>> Guile's input buffer is empty, it will try to refill the buffer using
>> the port type's 'fill_input' method.  For file ports this is
>> fports.c:fport_fill_input, and for custom binary ports it's
>> r6rs-ports.c:cbip_fill_input.
>
> In order to know if the port has at least one byte ready, you need to be
> able to call 'char-ready?', don't you? And as you mentioned, that
> doesn't work on custom binary ports. I'm still exploring this part of
> Guile so I might not have the details right.
>
>>
>> So again I ask, why use custom binary ports for sockets?
>>
>
> Thanks for the detailed explanation; I think I'll move away from
> them. Were they added just to meet the r6rs spec? Are there any good
> uses for custom binary ports that you're aware of?

Yes, custom ports are needed if the data that's being read is not coming
from a file, pipe, device, socket, string, or bytevector (the cases
covered by Guile's existing native port types; maybe I'm forgetting
some).  Similarly for writing.

For example, suppose you wanted a make an input port that reads
keypresses from a GUI interface.  In this case, the keypresses are
received via events during the main event loop.  There is no underlying
POSIX file descriptor to read from, nor is there a single Scheme string
or bytevector.  You could use pipes I suppose, but that assumes POSIX.
Another way is to make a custom port.

In the case of guile-xcb, I see two places where custom ports are used
in auth.scm.  Here's the first case:

--8<---cut here---start->8---
(define auth-file (open-file (getenv "XAUTHORITY") "rb"))

(define port
  (make-custom-binary-input-port
   "xcb-auth-input"
   (lambda (bv start count)
 (do ((n 0 (1+ n))
  (ch (read-char auth-file) (read-char auth-file)))
 ((or (>= n count) (eof-object? ch)) n)
   (bytevector-u8-set!
bv (+ start n) (char->integer ch
   #f #f (lambda () (close-port port
--8<---cut here---end--->8---

First of all, that '(close-port port)' is wrong.  Here you are trying to
tell Guile how to close 'port', and this just recurses.  I guess it
should be (close-port auth-file)'.

However, I don't see why this custom port is needed at all.  You've
already opened 'auth-file' in binary mode, so you should be able to use
it directly, i.e. replace the above code with:

(define port (open-file (getenv "XAUTHORITY") "rb"))

The other case is here (also in auth.scm):

--8<---cut here---start->8---
(define (read! bv start count)
  (let* ((in-bv (make-bytevector count))
 (bytes-read (recv! sock in-bv)))
(bytevector-copy! in-bv 0 bv start bytes-read)
bytes-read))
(define port (make-custom-binary-input-port "xcb-input" read! #f #f #f))
--8<---cut here---end--->8---

Again, I don't see why this custom port should be needed.
You should be able to use 'sock' directly.

Anyway, thanks for working on this! :)

 Regards,
   Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii  writes:

> I guess you didn't configure without threads on GNU/Linux, did you?
> If not, I suggest to try that, my impression is that Guile without
> threads is not used too much on Posix platforms.

Hydra, a continuous integration system, runs Guile's "make check" with
threads disabled on several POSIX platforms, so there's no need for
Panicz to do this test.

  http://hydra.nixos.org/jobset/gnu/guile-2-0#tabs-status

> Anyway, I looked into this a bit.  I can confirm that the simple
> program you mentioned the first time aborts due to "stack overflow"
[...]
> [...] it turns out that GC_get_stack_base, which
> is implemented in libgc, returns zero as the stack base.

Thanks for looking into this, Eli!

This raises the question: what's the relevant difference between
Panicz's simple 'main' and Guile's 'main' (in libguile/guile.c) that
causes one to (apparently) initialize the stack base properly, where the
other fails?  It would be worthwhile to find out.

Thanks,
  Mark

Re: [ANN] Guile-WM 0.1/Guile XCB 1.1

[Mark H Weaver]

mark.d.wit...@gmail.com writes:

> Mark H Weaver  writes:
>
>> * Although the README says that Guile 2.0.9 is required, the configure
>>   script will accept nothing less than Guile 2.2.  Since 2.2 does not
>>   exist yet, it would be good to accept Guile 2.0 in the meantime :)
>
> Yes, that's a mistake on my part. I've pushed the necessary change the
> configure.ac.

Excellent, thanks!

>> * After changing 2.2 -> 2.0, the next problem is that the build process
>>   seems to assume that (xcb xml) is already in Guile's %load-path, but
>>   the build system doesn't arrange for this.  I worked around this by
>>   setting GUILE_LOAD_PATH to the build directory.
>
> This one has me a bit confused; I compiled it all right with
> GUILE_LOAD_PATH unset. Can you send me the output from running make?

Here's what happens on my Debian wheezy system, starting from a pristine
git checkout (using "git clean -dxf"):

--8<---cut here---start->8---
mhw@tines:~/guile-xcb$ ./configure && make
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for pkg-config... (cached) /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
configure: checking for guile 2.0
configure: checking for guile 2.2
configure: found guile 2.2
checking for Guile site directory... /usr/local/share/guile/site/2.0
checking for guile... /usr/local/bin/guile
checking for guild... /usr/local/bin/guild
checking for guile-config... /usr/local/bin/guile-config
checking Guile lib directory... /usr/local/lib
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
restore=: && backupdir=".am$$" && \
am__cwd=`pwd` && CDPATH="${ZSH_VERSION+.}:" && cd . && \
rm -rf $backupdir && mkdir $backupdir && \
if (/bin/bash /home/mhw/guile-xcb/build_aux/missing makeinfo --version) 
>/dev/null 2>&1; then \
  for f in guile-xcb.info guile-xcb.info-[0-9] 
guile-xcb.info-[0-9][0-9] guile-xcb.i[0-9] guile-xcb.i[0-9][0-9]; do \
if test -f $f; then mv $f $backupdir; restore=mv; else :; fi; \
  done; \
else :; fi && \
cd "$am__cwd"; \
if /bin/bash /home/mhw/guile-xcb/build_aux/missing makeinfo   -I . \
 -o guile-xcb.info guile-xcb.texi; \
then \
  rc=0; \
  CDPATH="${ZSH_VERSION+.}:" && cd .; \
else \
  rc=$?; \
  CDPATH="${ZSH_VERSION+.}:" && cd . && \
  $restore $backupdir/* `echo "./guile-xcb.info" | sed 's|[^/]*$||'`; \
fi; \
rm -rf $backupdir; exit $rc
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
flow/event-loop.scm --output=flow/event-loop.go
wrote `flow/event-loop.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
xcb/xml/records.scm --output=xcb/xml/records.go
wrote `xcb/xml/records.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
xcb/xml/enum.scm --output=xcb/xml/enum.go
wrote `xcb/xml/enum.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
xcb/xml/type.scm --output=xcb/xml/type.go
wrote `xcb/xml/type.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
language/xml-xcb/support.scm --output=language/xml-xcb/support.go
wrote `language/xml-xcb/support.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
xcb/xml/union.scm --output=xcb/xml/union.go
wrote `xcb/xml/union.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
xcb/xml/struct.scm --output=xcb/xml/struct.go
wrote `xcb/xml/struct.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
xcb/xml/doc.scm --output=xcb/xml/doc.go
wrote `xcb/xml/doc.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
xcb/xml/connection.scm --output=xcb/xml/connection.go
wrote `xcb/xml/connection.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
language/xml-xcb/compile-scheme.scm --output=language/xml-xcb/compile-scheme.go
wrote `language/xml-xcb/compile-scheme.go'
GUILE_FLAGS="-C /home/mhw/guile-xcb" /usr/local/bin/guild compile 
language/xml-xcb/spec.scm --output=language/xml-xcb/spec.go
wrote `

Re: [ANN] Guile-WM 0.1/Guile XCB 1.1

[Mark H Weaver]

m...@markwitmer.com writes:

> Mark H Weaver  writes:
>
>> Here's what happens on my Debian wheezy system, starting from a pristine
>> git checkout (using "git clean -dxf"):
>> ...
>> ice-9/boot-9.scm:106:20: In procedure #> ice-9/boot-9.scm:97:6 (thrown-k . args)>:
>> ice-9/boot-9.scm:106:20: no code for module (xcb xml)
>> make: *** [xcb/event-loop.go] Error 1
>> mhw@tines:~/guile-xcb$ 
>
> Thanks for sending this. That was another file placed out of order in
> Makefile.am.  I've pushed a fix for that as well now and did a complete
> make uninstall/make/make install and it looks all right. I believe
> (fingers crossed!) it should build correctly if you pull the latest
> changes.

That fixes the build for me.

Thanks!
  Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii  writes:

>> From: Mark H Weaver 
>> Cc: Panicz Maciej Godek ,  guile-user@gnu.org
>> Date: Sun, 25 Aug 2013 12:59:43 -0400
>> 
>> Eli Zaretskii  writes:
>> 
>> > I guess you didn't configure without threads on GNU/Linux, did you?
>> > If not, I suggest to try that, my impression is that Guile without
>> > threads is not used too much on Posix platforms.
>> 
>> Hydra, a continuous integration system, runs Guile's "make check" with
>> threads disabled on several POSIX platforms, so there's no need for
>> Panicz to do this test.
>
> Sorry, I disagree.  "make check" is not the issue here: Guile passes
> it on my machine with flying colors (see my reports back then).  And
> yet a simple program shown by Panicz aborts due to "stack overflow".
> I was suggesting to configure Guile on GNU/Linux without threads and
> run that same program: perhaps it will also fail.

Okay, I stand corrected.

 Thanks,
   Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii  writes:

>> From: Mark H Weaver 
>> Cc: Eli Zaretskii ,  "guile-user\@gnu.org" 
>> Date: Fri, 23 Aug 2013 11:13:19 -0400
>> 
>> Of course, ideally we would allow the entire Guile install to be freely
>> relocatable, as Windows users have come to expect.  This would require
>> some help from a Windows developer.
>
> Why should this feature be limited to Windows builds of Guile?  Why
> not make it work for Posix platforms as well?  Some other projects
> already do, they use argv[0] and PATH search to find the place where
> the executable is installed, and record relative file names from
> there.

Well, that's not generally how things are done on POSIX systems, and
furthermore I don't see how this could be made to work reliably on
POSIX.  The location of the Guile executable is sometimes not found in
either argv[0] or PATH.  For example, if you run the following script:

--8<---cut here---start->8---
#!/usr/local/bin/guile -s
!#

(format #t "~s~%" (program-arguments))
--8<---cut here---end--->8---

argv[0] will point to the script, not the Guile executable.  I suppose
we could detect if argv[0] points to a script and read the shebang (and
iterate, if the shebang points to another script), but even that doesn't
cover all cases.  It's possible to pass any arbitrary value for argv[0]
using the 'exec' family of system calls.

Regards,
  Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Mark H Weaver  writes:

> Eli Zaretskii  writes:
>
>>> From: Mark H Weaver 
>>> Cc: Eli Zaretskii ,  "guile-user\@gnu.org" 
>>> 
>>> Date: Fri, 23 Aug 2013 11:13:19 -0400
>>> 
>>> Of course, ideally we would allow the entire Guile install to be freely
>>> relocatable, as Windows users have come to expect.  This would require
>>> some help from a Windows developer.
>>
>> Why should this feature be limited to Windows builds of Guile?  Why
>> not make it work for Posix platforms as well?  Some other projects
>> already do, they use argv[0] and PATH search to find the place where
>> the executable is installed, and record relative file names from
>> there.
>
> Well, that's not generally how things are done on POSIX systems, and
> furthermore I don't see how this could be made to work reliably on
> POSIX.  The location of the Guile executable is sometimes not found in
> either argv[0] or PATH.  For example, if you run the following script:
>
> #!/usr/local/bin/guile -s
> !#
>
> (format #t "~s~%" (program-arguments))
>
> argv[0] will point to the script, not the Guile executable.  I suppose
> we could detect if argv[0] points to a script and read the shebang (and
> iterate, if the shebang points to another script), but even that doesn't
> cover all cases.  It's possible to pass any arbitrary value for argv[0]
> using the 'exec' family of system calls.

I forgot to mention a much more common case that would cause problems.

Remember that Guile is a library, not just an executable.  So argv[0]
could point to any arbitrary executable that's linked with libguile.
There are many different mechanisms available to do the linking,
including static linking, LD_LIBRARY_PATH, rpaths, etc.  Trying to
portably figure out which copy of libguile was used, and where it is
located, is a can of worms, to put it mildly.

Having said that, we should probably come up with a solution for
relocatable builds on MacOS X as well.

Regards,
  Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

l...@gnu.org (Ludovic Courtès) writes:

> Eli Zaretskii  skribis:
>
>>> From: Mark H Weaver 
>>> Cc: Eli Zaretskii ,  "guile-user\@gnu.org" 
>>> 
>>> Date: Fri, 23 Aug 2013 11:13:19 -0400
>>> 
>>> Of course, ideally we would allow the entire Guile install to be freely
>>> relocatable, as Windows users have come to expect.  This would require
>>> some help from a Windows developer.
>>
>> Why should this feature be limited to Windows builds of Guile?  Why
>> not make it work for Posix platforms as well?  Some other projects
>> already do, they use argv[0] and PATH search to find the place where
>> the executable is installed, and record relative file names from
>> there.
>
> This is not completely reliable when using argv[0] because argv[0] can
> be anything, though this can be work around on Linux-based systems with
> /proc/self/exe as done in
> <http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/guile-relocatable.patch>.

This finds the executable, but as I noted in a recent message, that
doesn't reliably lead to finding the location of libguile.  I think
you're assuming that the executable is the standard 'guile' executable,
or at least that it's installed in the same prefix as libguile.

> Here’s a proposal: add a --enable-relocatable configure option, have it
> default to yes on MinGW and Cygwin (assuming there’s a method that works
> reliably on these), and default to no elsewhere; alternately, have it
> default to no everywhere.
>
> WDYT?

In theory, I like this idea.  I just don't see how it can be made to
work reliably on POSIX systems.

Thanks,
  Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii  writes:

>> From: Mark H Weaver 
>> Cc: godek.mac...@gmail.com,  guile-user@gnu.org
>> Date: Sun, 25 Aug 2013 15:56:53 -0400
>> 
>> Remember that Guile is a library, not just an executable.  So argv[0]
>> could point to any arbitrary executable that's linked with libguile.
>
> We can provide an API for passing to the library the root of its
> installation.

I suppose, but that assumes that the main program knows the location of
the libguile installation it's linked to.  How would it know this?  In
the common case on POSIX, the dynamic linker takes care of finding a
suitable copy of libguile, drawing from sources such as /etc/ld.so.conf,
/etc/ld.so.conf.d/*, LD_LIBRARY_PATH, rpaths, etc.  How can the main
program know?

> And btw, how is this different from GCC looking for its libgcc or GDB
> looking for its Python scripts?

GCC and GDB are programs, not libraries.  Finding out the location of
the current executable is a much easier problem than finding out the
install prefix of a particular library.

> An executable linked with libguile will either be in /usr/bin or
> somesuch, i.e. close to /usr/lib where libguile lives; or it will be
> in some random place under the user's home directory, in which case
> either libguile is in the default place, or it is near the binary.
> The latter case is precisely the additional feature where looking for
> the library nearby will be a benefit.

You're making a lot of very dubious assumptions here.  I'm uncomfortable
advertising a new feature for Guile that is impossible to implement
robustly.  If it has to make assumptions such as "the libguile library
is probably near the binary", it is likely to fail in many cases.

> It's true that this is not 100% bulletproof on Posix, but it's close.

So far, I've not seen a solution that's anywhere near close to being
correct on POSIX.  All I've seen is a bunch of very dubious guesses.

> And when it doesn't work, we didn't lose anything, we are just back to
> where we are now, right?

I disagree.  If we advertise a new feature that cannot work without
making dubious assumptions, then we're making promises we can't keep.
That's a step in the wrong direction, IMO.

Regards,
  Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii  writes:

>> Date: Sun, 25 Aug 2013 19:24:17 -0400
>> From:  
>> Cc: guile-user@gnu.org
>> 
>> On windows, I think you call GetModuleFileName() with the handle that was 
>> passed to DllMain().
>> Is it possible for libguile to do that?
>
> I see no reasons why not.

Yes, that much we can agree on.  Since the Windows API provides a
reliable method to tell us the location of the libguile library, we can
certainly use that to find the rest of the Guile installation.

>> On a POSIXy system, there are shared places for config files,
>> libraries, documentation, and so on.  It's just different.  I don't
>> see why we would need to be "relocatable" here.
>
> One reason to be "relocatable" is to be able to easily run
> applications from the build or source tree, while a different version
> is installed on the system.

We already have a solution for this.  Just run "meta/guile" from the
build directory, or more generally "meta/uninstalled-env".

>> From: Mark H Weaver 
>> Cc: godek.mac...@gmail.com,  guile-user@gnu.org
>> Date: Sun, 25 Aug 2013 17:42:27 -0400
>> 
>> Eli Zaretskii  writes:
>> 
>> > We can provide an API for passing to the library the root of its
>> > installation.
>> 
>> I suppose, but that assumes that the main program knows the location of
>> the libguile installation it's linked to.  How would it know this?
>
> We are talking about the situation where libguile is _not_ installed
> in the usual places.  Why would a program _not_ know where that is?

I explained that in the part of my message that you omitted from your
quotation.

I can think of one notable case where the program would know where
libguile is located: when the program bundles its own copy of Guile, and
therefore assumes that libguile and the program are always moved
together as an atomic unit.

This practice is frowned upon on most POSIX systems (and rightfully so),
but I guess it's fairly common on Windows and MacOS X systems.

In cases where libguile is installed independently, and where libguile
has been moved since the program was built, there's no reason why the
program would know where libguile is located.  Every time the program is
run, the dynamic linker searches for libguile in many different places.
The set of places it looks depends on the platform.

>> > And btw, how is this different from GCC looking for its libgcc or GDB
>> > looking for its Python scripts?
>> 
>> GCC and GDB are programs, not libraries.  Finding out the location of
>> the current executable is a much easier problem than finding out the
>> install prefix of a particular library.
>
> The issue is how to find the auxiliary files _given_ the location of
> the executable, not how to find where the executable itself lives.

Yes, and those executables (GCC and GDB) can reasonably assume that
their auxilliary files are always installed in the same prefix as the
executables, since they are part of the same package.

However, libguile and some-program-that-uses-libguile are _not_ part of
the same package.  There's no reason to assume that they'll be installed
in the same prefix, and they could be moved independently of each other.

When the program is compiled, of course it needs to know where libguile
is located, but since we're talking about making these packages
relocatable, there's no guarantee that libguile will be in the same
place as it was when the program was compiled.

>> > And when it doesn't work, we didn't lose anything, we are just back to
>> > where we are now, right?
>> 
>> I disagree.  If we advertise a new feature that cannot work without
>> making dubious assumptions, then we're making promises we can't keep.
>> That's a step in the wrong direction, IMO.
>
> My turn to disagree.

Okay, again: How exactly do you suggest we determine the location of the
libguile installation from an arbitrary executable that links to it?

 Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

I wrote:

> I can think of one notable case where the program would know where
> libguile is located: when the program bundles its own copy of Guile, and
> therefore assumes that libguile and the program are always moved
> together as an atomic unit.
>
> This practice is frowned upon on most POSIX systems (and rightfully so),
> but I guess it's fairly common on Windows and MacOS X systems.

For what it's worth, I _do_ think that we should support this use case,
and therefore I agree that it would probably be useful to have the API
you suggested, where the program can tell libguile where libguile is
located.

However, the only case where this API could reasonably be used is when
libguile is bundled with the program in question, because that's the
only case where the program would reliably know the location of the
libguile that the dynamic linker found.

Regards,
  Mark

Re: SRFI-105: syntax error/warning

[Mark H Weaver]

Alexandru Cojocaru  writes:

> when using the SRFI-105 syntax everything works fine, but I receive
> syntax warnings
>
> bash> cat 1.scm
> (read-enable 'curly-infix)
> (define a {1 + 1}) (newline)
> (display a) (newline)

The problem is that the expression "(read-enable 'curly-infix)" is not
evaluated until after the entire file has been read and compiled, and
that's too late.

Please remove "(read-enable 'curly-infix)", and in its place put
"#!curly-infix", as was recommended in the SRFI-105 section of the
manual.  You should also put a blank line before it, so that it's not
interpreted as the shebang at the top of a script.

 Regards,
   Mark

Re: Emacsy: Context Sensitive Commands Design Question

[Mark H Weaver]

Shane Celis  writes:

> 6. Tag commands as special procedures perhaps by adding something to
> their procedure properties.  Implement a "command?" procedure.  Export
> commands and procedures to the same module. Then just pluck the
> commands out of all the procedures by using command?.

This is closest to what Emacs does, and probably the best solution, IMO.  See

which describes the 'commandp' predicate, whose definition is in eval.c.

Regards,
  Mark

Re: How to read integers from file faster?

[Mark H Weaver]

Darren Hoo  writes:
> It is way too slow to read numbers from a file simply by using `read' 

Indeed.  Looking at the code, I can see a lot of room for improvement,
especially in 'string->number' (which is used also by 'read').  I've
added an item to my TODO list to optimize this.  Hopefully by 2.0.10 or
2.0.11, it will be faster.

> Are there any primitives in Guile that is equivalent to C's scanf?

I'm afraid there's currently nothing that will read integers very fast.
I experimented with using the Dynamic FFI to wrap various C functions,
but was unable to achieve more than a 2x improvement this way.

In the meantime, please remember that part of Guile's philosophy
(inherited from Emacs) is to use C extensions where necessary for
performance critical bits.  For the basics of how to do this, see:

  https://www.gnu.org/software/guile/manual/html_node/C-Extensions.html

As an example, I've attached a small C extension that uses 'fscanf' to
read integers from a file until EOF is reached, and returns them as a
list.  Here's how to compile and load it on GNU/Linux:

--8<---cut here---start->8---
mhw@tines:~$ gcc -shared -o read_integers_from_file.so -fPIC 
read_integers_from_file.c $(pkg-config --cflags guile-2.0)
mhw@tines:~$ guile
GNU Guile 2.0.9
Copyright (C) 1995-2013 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (load-extension "./read_integers_from_file.so" 
"scm_init_read_integers_from_file")
scheme@(guile-user)> (use-modules (read-integers-from-file))
scheme@(guile-user)> (define lst (read-integers-from-file "rnd.txt"))
--8<---cut here---end--->8---

Note that although this code does partial error checking, it cannot cope
with integers that don't fit in a C long, and it also assumes that the
file is properly formatted.

  Regards,
Mark

#include 
#include 

static SCM
read_integers_from_file (SCM filename)
{
  char *fname = scm_to_locale_string (filename);
  FILE *f = fopen (fname, "r");
  SCM result = SCM_EOL;
  int err = 0;
  long num;

  if (f == NULL)
err = 1;
  else
{
  while (fscanf (f, "%dl", &num) == 1)
result = scm_cons (scm_from_long (num), result);
  if (ferror (f))
err = 1;
  if (fclose (f))
err = 1;
}
  free (fname);
  if (err)
scm_syserror ("read-integers-from-file");
  return scm_reverse_x (result, SCM_EOL);
}

void
init_read_integers_from_file (void *unused)
{
  scm_c_define_gsubr ("read-integers-from-file", 1, 0, 0,
  read_integers_from_file);
  scm_c_export ("read-integers-from-file", NULL);
}

void
scm_init_read_integers_from_file ()
{
  scm_c_define_module ("read-integers-from-file",
   init_read_integers_from_file, NULL);
}

Re: [Denemo-devel] Problem with wide characters on upgrading to guile 2.x

[Mark H Weaver]

Richard Shann  writes:

> On Tue, 2013-09-03 at 19:53 +0200, Thien-Thi Nguyen wrote:
> [...]
>>pt_BR.utf8 is not a supported locale on your system.
>
> yes, that is why I chose it, so as to trigger the error. It is graceful
> handling of this error that I was after - it should default to
> untranslated, not abort the program.

I disagree.  If Guile does not know what encoding to use, that's a
serious error that should not be ignored, at least not by default.
Please keep in mind that Guile is used for many diverse tasks, including
scripts that are run when the user's not looking, e.g. cron jobs.  Guile
must not silently corrupt data, which could easily happen if it silently
ignores the error without knowing what character encoding to use.

For fully interactive programs, ignoring the error and defaulting to the
C locale (e.g. ASCII) is more reasonable, because any corruptions will
hopefully be noticed by the user.  To do that, please consider the code
that Ludovic suggested:

  (catch 'system-error
  (lambda ()
(setlocale LC_ALL ""))
  (lambda args
(format (current-error-port)
"warning: failed to install locale: ~a~%"
(strerror (system-error-errno args

However, it should be noted that if you do this, Guile 2 (unlike 1.8)
will still raise an error later if you try to read a byte outside of the
ASCII range, or if you try to write a non-ASCII character.

Note that Guile 1.8 conflated characters and bytes, like many older
non-i18n programs, and thus effectively chose Latin-1 by default.
However, since most modern GNU systems use UTF-8 by default (at least
outside of the CJK world) this guess is most likely wrong, and thus
likely to silently corrupt non-ASCII characters.

 Regards,
   Mark

Re: is (web client) ready for use even for the simplest task?

[Mark H Weaver]

Darren Hoo  writes:

> (use-modules (web client))
>
> (http-post "http://www.google.com/";) 
>
> the POST request is sent without the Content-Length header

RFC 2616 makes it clear that the Content-Length header is optional.

> OK, let's add something to the body
>
> (http-post "http://www.google.com/"; #:body "")
>
> Howcome the request now becomes an http GET request:
>
> GET / HTTP/1.1
> Content-Type: text/plain;charset=utf-8
> Host: www.google.com
> Connection: close

I just applied a fix for this to the stable-2.0 branch in git.  In the
meantime, the workaround is to explicitly pass a content-type header
that specifies the charset, like this:

(http-post "http://www.google.com/";
   #:body ""
   #:headers '((content-type text/plain (charset . "utf-8"

> This is really ridiculous.

You found a bug.  It happens.  There's no need to be obnoxious about it.

  Mark

Re: is (web client) ready for use even for the simplest task?

[Mark H Weaver]

Mark H Weaver  writes:

>> GET / HTTP/1.1
>> Content-Type: text/plain;charset=utf-8
>> Host: www.google.com
>> Connection: close
>
> I just applied a fix for this to the stable-2.0 branch in git.  In the
> meantime, the workaround is to explicitly pass a content-type header
> that specifies the charset, like this:
>
> (http-post "http://www.google.com/";
>#:body ""
>#:headers '((content-type text/plain (charset . "utf-8"

Sorry, this wasn't quite enough if the body is non-empty.  In order to
work around the bug, you also need to explicitly pass the content-length
header, like this:

  (use-modules (web client) (rnrs bytevectors))

  (let ((bv (string->utf8 "test")))
(http-post "http://www.google.com/";
   #:body bv
   #:headers `((content-type text/plain (charset . "utf-8"))
   (content-length . ,(bytevector-length bv)

More generally, when using (web client) for anything other than GET, you
must explicitly provide a content-type header with charset, and also a
content-length (if appropriate).

The bug is fixed in stable-2.0 git.

  Mark

Re: is (web client) ready for use even for the simplest task?

[Mark H Weaver]

Mark H Weaver  writes:

>> (http-post "http://www.google.com/";) 
>>
>> the POST request is sent without the Content-Length header
>
> RFC 2616 makes it clear that the Content-Length header is optional.

Sorry, I should have qualified this statement with "in this case",
i.e. when there's no body.  (web client) does add a Content-Length
header when there's a non-empty body.

It should also add a Content-Length header when an empty body is
provided, but it doesn't.  I'll fix this soon.  Anyway, the work-around
I provided adds a Content-Length header explicitly, so it works around
this problem too.

For the record, I didn't write this code; I'm just fixing it.

 Mark

Re: are double cells safe to use?

[Mark H Weaver]

Roland Orre  writes:

> I consider using double cells for efficient implementation of binary trees.
>
> Is this safe? (i.e. no pre-assumptions made somewhere what these are used for)
>
> For prototyping I intended to add
> double-cons
> set-cbr!
> set-ccr!
> cbr
> ccr
>
> to the scheme level. Is this safe?

No, this won't work.  First of all, double cells cannot be used to store
four SCM values, because the first word of a double cell has to contain
the type tag.  Pairs avoid the type tag by the clever hack: the low bit
of every SCM value is 0, so if the first word of a heap object has 0 in
the low bit, that heap object is assumed to be a pair.  But this means
that every other heap object must start with a word whose low bit is 1.

So the best you could do with a double cell would be to store three SCM
objects, which is no better space efficiency than you already get with a
vector of size 3 (in the master branch, which will become Guile 2.2).

Another problem with creating yet another new fundamental data type is
that in order to use it efficiently, we'd need to create more VM
instructions to access it.  That means more opcodes from our limited
8-bit opcode space, and more code in the VM, which is bad because
ideally a VM should be compact for good cache behavior.

I think you should just use records, or maybe vectors, for this.

 Mark

Re: cond-expand-provide threads?

[Mark H Weaver]

Hi,

Panicz Maciej Godek  writes:
> is there any way to check (most preferably at the cond-expand stage)
> whether thread support is enabled in guile?

  (provided? 'threads)

However, this is a run-time check, not a compile-time check.  It has to
be this way, because a scheme module could be compiled to a .go file,
and then this same module could be used by both threaded and unthreaded
versions of libguile.

  Mark

Re: [ANN] Guile-WM 0.1/Guile XCB 1.1

[Mark H Weaver]

Hi Mark,

Shortly after the releases of Guile XCB 1.1 and Guile WM 0.1, some
important problems were fixed in the build system.  Would you be willing
to produce new tarball releases?

Several of us are interested in Guile WM and would like to make it
easier for new users to play with and hack on.  Toward that end, we'd
like to package it for GNU Guix , which has a
policy of using only upstream tarballs.

Thanks!
  Mark

Re: Where is the backtrace?

[Mark H Weaver]

Panicz Maciej Godek  writes:

> I've checked this with other implementations. Racket, Gambit and Biwa
> Scheme were equally uninformative. Kawa's backtrace was a nightmare to
> me, but perhaps someone with more knowledge would be able to infer the
> actual location of the error. Actually, the only implementation that
> exposed the exact location was Chicken. It proves that there must be a
> way to track this information in spite of TCO.

I suspect the reason Chicken kept the information is that in Chicken,
IIUC, tail calls are just normal C function calls, so the stack fills
until it reaches a certain size and then is garbage collected.

The solution I favor is that of MIT Scheme, which records the backtrace
as a chain of rings.  Each time a tail call is done, an entry is
recorded in the topmost ring, so between every two non-tail calls, up to
N tail calls are preserved.  If more than N tail calls are done in a
row, then the older ones are forgotten.

Hopefully we'll have this for Guile some day.

 Mark

Re: libguile thread safety

[Mark H Weaver]

Chris Vine  writes:
> I am having problems using libguile in a multi-threaded environment,
> which boils down to the following small test case, which fails with a
> segmentation fault with guile-2.0.9:

I've recently noticed that the module loading process in Guile is not
thread safe.  I hope to fix this in the next month or two, but for now
I'd recommend that one thread should initialize Guile and load the
modules that will be needed, before allowing multiple threads to enter
Guile mode.

Regards,
  Mark

Re: libguile thread safety

[Mark H Weaver]

Panicz Maciej Godek  writes:

> 2014/1/4 Chris Vine :
>> It seems as if top level variables in code evaluated by scm_c_eval_string()
>> are shared between concurrently running threads.  Is this really the
>> intended behaviour (it is a significant and unexpected usability
>> restriction)?  Maciej (I hope that is your first name) can you reproduce
>> this?
>
[...]
>
> It indeed does seem that the threads share their top-level bindings on
> guile's side, and I suppose that this is the intended behaviour. I
> think that it can be easily adjusted with scm_eval_string_in_module,
> i.e. if you provide a separate module for each thread.

Indeed, top-level bindings are always looked up in the "current module".
Each thread has its own "current module", but 'scm_with_guile' initially
sets the current module to (guile-user).  That's usually desirable,
because you may have bound your own application-specific procedures and
global variables in (guile-user), and you want all threads to have
access to those by default.

The usual way to make thread-local variables in Guile is to use
parameters[1] or fluids[2].  It's rather unusual to create fresh modules
for each thread, but if you really want to do that you can start each
thread by evaluating "(set-current-module (make-fresh-user-module))".

[1] API Reference > Scheduling > Parameters (section 6.21.8)
[2] API Reference > Scheduling > Fluids and Dynamic States (section 6.21.7)

  Mark

Re: libguile thread safety

[Mark H Weaver]

Chris Vine  writes:
> Unfortunately, from what Mark Weaver says, the module system doesn't
> initialize itself in a thread safe way either :(.

That's not quite right.  It initializes itself in a thread safe way.
However, _after_ initialization, it does not load modules in a thread
safe way.

> That is more problematic because you might not know in advance what
> modules are to be loaded by any particular code.  The task model isn't
> suppose to import cross-dependencies of that kind.

I can see that this is a serious problem, and I'm going to focus on
fixing it properly and soon, hopefully in the next two weeks.

 Mark

Re: libguile thread safety

[Mark H Weaver]

Panicz Maciej Godek  writes:

> 2014/1/5 Chris Vine :
>
>> I actually have a prototype working well now, apart from the module
>> issue Mark Weaver referred to, which I have to say has not yet bitten me
>> but no doubt might.
>
> If you are worried about this, you could try to take your own
> precautions and redefine the "use-modules" syntax (or others, like
> "resolve-module", if you are planning to use them) to make sure that
> it is thread safe.
>
> The sole act of redefinition would require you to create a new module like 
> this:
> (define-module (safe-modules)
>   #:use-module (ice-9 format)
>   #:replace ((safe-use-modules . use-modules)))
>
> (define-syntax-rule (safe-use-modules modules ...)
>   (begin
> (format #t "loading modules ~a... " '(modules ...))
> (use-modules modules ...)
> (format #t "done!\n")))
>
> You'd need to replace the the messages with some thread
> synchronization mechanism, to make sure that if one thread calls
> "use-modules", all the others that would try to do that at that time
> would have to wait.

This wouldn't work.  First of all, 'use-modules' is a macro, not a
procedure, but more importantly, modules are autoloaded in many places
deep in the guts of the system.  Notice that Chris' test cases didn't
use 'use-modules' at all.  In fact, a module is autoloaded the first
time 'scm_c_eval_string' is called.  Also, there are potential
deadlocks.

We should just fix the problem properly, and that's what I intend to do.

 Mark

Re: Filename encoding

[Mark H Weaver]

Chris Vine  writes:

> A number of guile's scheme procedures look-up or reference files on a
> file system (open-file, load and so forth).
>
> How does guile translate filenames from its internal string
> representation (ISO-8859-1/UTF-32) to narrow string filename encoding
> when looking up the file?  Does it assume filenames are in locale
> encoding (not particularly safe on networked file systems) or does it
> provide a fluid for this?  (glib caters for this with the
> G_FILENAME_ENCODING environmental variable.)

It assumes filenames are in locale encoding.  Ditto for virtually
everything that interfaces with POSIX-style byte strings, including
environment variables, command-line arguments, etc.  Encoding errors
will raise exceptions by default.

My hope is that this will become less of an issue over time, as systems
increasingly standardize on UTF-8.  I see no other good solution.

Thoughts?

 Mark

Re: Converting a part of byte vector to UTF-8 string

[Mark H Weaver]

Panicz Maciej Godek  writes:

> Your solution seems reasonable, but I have found another way, which
> lead me to some new problems.
> I realised that since sockets are ports in guile, I could process them
> with the plain "read" (which is what I have been using them for
> anyway).
>
> However, this approach caused some new problems. The thing is that if
> I'm trying to read some message from port, and that message does not
> end with a delimiter (like a whitespace or a balancing, closing
> parenthesis), then the read would wait forever, possibly gluing its
> arguments.
>
> The solution I came up with is through soft ports. The idea is to have
> a port proxy, that -- if it would block -- would return an eof-object
> instead.

This is terribly inefficient, and also not robust.  Guile's native soft
ports do not support efficient reading, because everything is one
character at a time.  Also, Guile's 'char-ready?' currently does the job
of 'u8-ready?', i.e. it only checks if a _byte_ is available, not a
whole character, so the 'read-char' might still block.  Anyway, if this
is a socket, what if the data isn't available simply because of network
latency?  Then you'll generate a spurious EOF.


To offer my own answer to your original question: R7RS-small provides an
API that does precisely what you asked for.  Its 'utf8->string'
procedure accepts optional 'start' and 'end' byte positions.  I
implemented this on the 'r7rs-wip' branch of Guile git as follows:

http://git.savannah.gnu.org/gitweb/?p=guile.git;a=blob;f=module/scheme/base.scm;h=f110d4c2b241ec0941b4223cece05c309db5308a;hb=r7rs-wip#l327

  (import (rename (rnrs bytevectors)
  (utf8->string  r6rs-utf8->string)
  (string->utf8  r6rs-string->utf8)
  (bytevector-copy   r6rs-bytevector-copy)
  (bytevector-copy!  r6rs-bytevector-copy!)))

  [...]

  (define bytevector-copy
(case-lambda
  ((bv)
   (r6rs-bytevector-copy bv))
  ((bv start)
   (let* ((len (- (bytevector-length bv) start))
  (result (make-bytevector len)))
 (r6rs-bytevector-copy! bv start result 0 len)
 result))
  ((bv start end)
   (let* ((len (- end start))
  (result (make-bytevector len)))
 (r6rs-bytevector-copy! bv start result 0 len)
 result

  (define utf8->string
(case-lambda
  ((bv) (r6rs-utf8->string bv))
  ((bv start)
   (r6rs-utf8->string (bytevector-copy bv start)))
  ((bv start end)
   (r6rs-utf8->string (bytevector-copy bv start end)

Re: negating a number

[Mark H Weaver]

Richard Shann  writes:

> Is there something neater than
>
> (- 0 amount)
>
> to negate a number? Can't spot anything in the manual ...

Yes: (- amount)

Mark

Re: string-match error

[Mark H Weaver]

Ariel Rios  writes:

> Hello y'all,
>
> I am having problems using string-match. I have created a small script
> that calls string-match and I get:
>
> /home/ariel/console/mce/tests/./panda.scm:4:9: In procedure
> module-lookup: Unbound variable: string-match

As it says in the "Regular Expressions" section of the manual:

  "Before using the described functions, you should load this
   module by executing `(use-modules (ice-9 regex))'."

> If I start guile and then use string-match there is no problem:
> scheme@(guile-user)> (string-match "a" "a")
> $1 = #("a" (0 . 1))

It seems that the REPL starts with (ice-9 regex) loaded by default.

Regards,
  Mark

Re: Filename encoding

[Mark H Weaver]

Eli Zaretskii  writes:

>> From: Mark H Weaver 
>> Date: Wed, 15 Jan 2014 13:14:39 -0500
>> Cc: guile-user@gnu.org
>> 
>> My hope is that this will become less of an issue over time, as systems
>> increasingly standardize on UTF-8.  I see no other good solution.
>> 
>> Thoughts?
>
> MS-Windows filesystems will not standardize on UTF-8 in any observable
> future.

Well, I understand that MS has standardized on UTF-16 (right?) but what
matters from Guile's perspective is the encoding used by the POSIX-style
interfaces that Guile uses, such as 'open'.  Do you know what encoding
that is on Windows?

> Likewise, in some Far Eastern cultures, non-UTF encoding are still
> widely used.

*nod*

> An "other good solution" is to decode file names into Unicode based
> representation (which can be UTF-8) for internal handling, then encode
> them back into the locale-specific encoding when passing them to
> system calls and library functions that receive file names.  This is
> what Emacs does.

That's what Guile does too.  Internally, all strings are Unicode.  At
present we use either Latin-1 or UTF-32, but I intend to change the
internal representation to UTF-8 at some point.

   Thanks,
 Mark

Re: Filename encoding

[Mark H Weaver]

Chris Vine  writes:
> POSIX system calls are encoding agnostic.  The filename is just a series
> of bytes terminating with a NUL character.

Yes, I know, but conceptually these things are strings.  Unless you're
going to treat these filenames as black boxes to be copied from one
place to another but never manipulated, printed, or read, you need to
know their encoding and you need to treat them as strings internally.

> All guile needs to know is what encoding the person creating the
> filesystem has adopted in naming files and which it needs to map to.

Right, but how does it know that?  The closest thing we have to a
standard way to tell programs what encoding to use is via the locale.  I
believe that's what most existing internationalized programs do, anyway.

> So far as filenames are concerned, this seems to me to be something
> for which a fluid would be just the thing - it could default to the
> locale encoding but a user could set it to something else.

We could do that, but I'm not really sure how it would improve the
situation.  If Guile expects the program to know the encoding of
filenames on the filesystem, that just passes the buck to the program.
How does the program know what encoding to use?

Yes, the program can know the encoding if it's a custom program written
for one specific system.  However, if you write a program that's
supposed to work on any system, how do you know the encoding?

It seems to me that each system must standardize on a single encoding
for all filenames on that system, and the locale encoding is the defacto
standard way of telling programs what that is.

Regards,
  Mark

Re: Filename encoding

[Mark H Weaver]

Eli Zaretskii  writes:
> Windows stores file names on disk encoded in UTF-16, but converts them
> to the current codepage if you use Posix-style interfaces like 'open'
> and 'rename'.  (There are parallel APIs that accept UTF-16 encoded
> file names.)

Okay, so on Windows we should use the parallel APIs that accept UTF-16,
thus (apparently) completely avoiding the issue on that platform.

  Mark

Re: [PATCH v2 02/13] script language API for GDB: extension.[ch]

[Mark H Weaver]

Tom Tromey  writes:

>> "Ludovic" == Ludovic Courtès  writes:
>
> Ludovic> I guess this is another limitation of Guile’s current signal handling
> Ludovic> strategy, and something we should fix.
>
> FWIW I think it would be sufficient for gdb if scm_system_async_mark, or
> something like it, could be invoked from a signal handler.  Then a
> SIGINT in gdb could install an async callback that later checks gdb's
> quit flag.

As discussed on IRC, one way to accomplish this with current Guile is
as follows: establish a dedicated thread whose sole job it is to call
'scm_system_async_mark_for_thread' as directed by messages received from
a pipe.  A signal handler can then schedule asyncs by writing messages
to this pipe.

We'll try to come up with an nicer solution at some point.

 Mark

Re: Geting the value of a top level scheme value from C

[Mark H Weaver]

Richard Shann  writes:

> Given a C string that is the name of a Scheme variable what is the call
> I need to make from C to get the value?
>
> That is, in the Scheme I have
>
> (define foo "bar")
>
> Then in C I want to write
>
> SCM bar = scm_var_c_get_val ("foo");//imaginary function name

  SCM bar = scm_variable_ref (scm_c_lookup ("foo"));

For better efficiency, do this just once, after 'foo' has been defined:

  SCM foo_var = scm_c_lookup ("foo");

and henceforth just do:

  SCM bar = scm_variable_ref (foo_var);

 Mark

Re: Writing libraries for C programs using Guile Scheme

[Mark H Weaver]

Mateusz Kowalczyk  writes:
> I doubt that going from a single inherently unsafe but bloody fast
> language to slightly less unsafe but much slower language is an
> advantage here…

"Slightly less unsafe"?  Seriously?

> Do you feel like you can provide correctness proofs for your
> implementations of such security critical libraries? Scheme isn't
> exactly the safest language.

If you'd like to write a new TLS (or other widely-used security
critical) library in Haskell, along with formal and verifiable
correctness proofs, and that would be easy enough to use from programs
written in other languages such that it could actually make a viable
replacement for GnuTLS et al, that would be a great contribution to our
community, and I would *sincerely* thank you for it.

However, what I've mostly seen from you is negativity and FUD about
Scheme and Guile, both here and on IRC.  That is _not_ appreciated.

 Mark

Re: ‘with-exception-handler’ rationale

[Mark H Weaver]

Nikita Karetnikov  writes:

> I’ve expected ‘with-exception-handler’ to behave like ‘catch’, but it
> doesn’t.
>
> scheme@(guile-user)> ,use (srfi srfi-34)
> scheme@(guile-user)> (with-exception-handler (const "got it") (lambda () 
> (raise "boom!")))
> ice-9/boot-9.scm:106:20: In procedure # ice-9/boot-9.scm:97:6 (thrown-k . args)>:
> ice-9/boot-9.scm:106:20: Throw to key `srfi-34' with args `("boom!")'.
>
> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
> scheme@(guile-user) [1]> 
> scheme@(guile-user)> (catch 'misc-error (lambda () (error "boom!")) (const 
> "got it"))
> $1 = "got it"
>
> Is there an explanation that doesn’t involve diving into the land of
> continuations and dynamic wind?  I mean, when should I use the one but
> not the other?

Neil's response was exactly correct, but it used some terminology that
might be difficult for non-experts to understand, so I'm going to try my
hand at a simpler explanation.

When an exception is caught, there are two things that typically need to
be done:

1. Run the user-specified handler.

2. Do a non-local exit back to the catch/guard/with-exception-handler,
   returning the value produced by the handler.

'catch' and 'guard' handle both of these steps for you.

'with-exception-handler' is a more powerful but lower-level mechanism.
It handles step 1, but leaves step 2 to you.  If you use it, it is your
responsibility to handle the non-local exit yourself, typically by
invoking a continuation.  That's what the 'call/cc' and the call to 'k'
was about in Neil's response.

Having said all this, I think the shorter answer is that you should
probably be using 'guard' instead.

Regards,
  Mark

Re: Number in a module-name

[Mark H Weaver]

Diogo Ramos  writes:

> Is it possible to have a number as a module-name component?
>
> e.g. (define-module (42 bar))

At present, module name components must be symbols.  However, R7RS
allows either symbols or exact non-negative integers to be module name
components, so it seems likely that we'll add support for this at some
point.

  Mark

Re: libguile thread safety

[Mark H Weaver]

Chris Vine  writes:

> Has thread-safe module loading turned out to be too difficult to do, or
> is it still something that is likely to happen in the future?

I consider this to be an important issue, and I still intend to fix it.
However, fixing it in such a way that I could confidently believe would
not cause problems for existing users (who might do unusual things with
Guile's module system) is indeed non-trivial.

On the other hand, the lack of thread-safety in module loading doesn't
seem to be causing problems for most users in practice.  I suspect this
is because most programs load their modules during initialization, and
rarely (if ever) load modules afterwards.

Therefore, I'm considering fixing this in git master, which will become
Guile 2.2, and not in the stable-2.0 branch.

What do you think?

Mark

Re: GNU Guile 2.0.10 released

[Mark H Weaver]

l...@gnu.org (Ludovic Courtès) writes:

> Panicz Maciej Godek  skribis:
>
>> Delightful to hear that. Thanks a lot for the work!
>>
>> However, when I'm trying to build it under mingw, I get the following error:
>>
>> ../lib/.libs/libgnu.a(mkstemp.o): In function `mkstemp':
>> c:\dev\guile-2.0.10\lib/mkstemp.c:48: multiple definition of `mkstemp'
>> .libs/libguile_2.0_la-mkstemp.o:c:\dev\guile-2.0.10\libguile/mkstemp.c:68:
>> first defined here
>>
>> (and indeed, the function is defined twice, once in
>> libguile/mkstemp.c, and the second time in lib/mkstemp.c)
>
> Oops, indeed.
>
> Could you edit libguile/Makefile.am (or libguile/Makefile.in, if
> Automake isn’t installed), and remove occurrences of mkstemp.c in there?
>
> Let us know if it solves the problem.

Interesting.  Madsy and I both built Guile (950a966, very close to
2.0.10) successfully using MinGW without running into this problem.
I wonder why we didn't hit this.

 Mark

Re: Question on mutability of argument lists

[Mark H Weaver]

Marco Maggi  writes:

> Niels Möller wrote:
>> However,  for  a  natural implementation  which  evaluates
>> arguments and conses the results  up to a list, this seems
>> to require an  extra copy of the argument  list, after all
>> values are evaluated  and before the list is  bound to the
>> rest parameter.
>
> Yes,  but it  depends  on  what is  "natural"  for you.   In
> Vicare, when  calling a  Scheme function, all  the arguments
> are pushed on the Scheme stack segment (similar to calling a
> C  function) and  then the  function is  called; the  callee
> takes  the leftover  arguments and  puts them  into a  newly
> allocated  list.

Indeed, and Guile 2 also does this in compiled code.

 Mark

Re: Restricting eval

[Mark H Weaver]

Panicz Maciej Godek  writes:

> 2014-03-23 19:07 GMT+01:00 Grant Rettke :
>> On Sat, Nov 30, 2013 at 2:26 PM, Panicz Maciej Godek
>>  wrote:
>>> And hence my question: is there any way to restrict
>>> the execution environment of eval, e.g. to specify
>>> which symbols should be available? (For security
>>> reasons, I wouldn't want functions like "system"
>>> or "exit" to be present in that environment)
>>>
>>> Or perhaps there's some better way to do that?
>>
>> How did you end up achieving your goal?
>
> Oh, with Guile it turned out to be a piece of cake ;]
> It's thanks to first-class modules and the fact that a module can be
> provided as the second argument to eval.
> Guile actually has e.g. (ice-9 safe-r5rs), which exports a safe subset
> of Scheme, and (ice-9 null), which provides the most basic syntactic
> bindings and no functions whatsoever.
>
> So it is possible to either create a module in runtime using
> make-fresh-user-module and add all the necessary bindings, or to have
> some regular module prepared and obtain it using resolve-module.

It turns out that it's trivial to access any binding from any module,
even from code evaluated in a module with a restricted set of bindings.

  http://lists.gnu.org/archive/html/guile-devel/2012-05/msg00041.html

I regret calling it a "Psyntax security hole", since Guile has never
claimed to support secure sandboxing, but it's something to keep in
mind.

I'd like to add support for secure sandboxing in a future version of
Guile, but in the meantime you'd better run it in a VM or container if
that's what you want.

 Mark

Re: 'or' and multiple values

[Mark H Weaver]

Hi Neil,

Neil Jerram  writes:

> In the following, is the last result a bug?
>
>   GNU Guile 2.0.9-deb+1-1
>   Copyright (C) 1995-2013 Free Software Foundation, Inc.
>
>   Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
>   This program is free software, and you are welcome to redistribute it
>   under certain conditions; type `,show c' for details.
>
>   Enter `,help' for help.
>   scheme@(guile-user)> (values 'a 'b)
>   $1 = a
>   $2 = b
>   scheme@(guile-user)> (or (values 'a 'b))
>   $3 = a
>   $4 = b
>   scheme@(guile-user)> (or #f (values 'a 'b))
>   $5 = a
>   $6 = b
>   scheme@(guile-user)> (or (values 'a 'b) (values 'c 'd))
>   $7 = a
>
> In other words it seems 'or' doesn't propagate multiple values in a
> non-tail position.  Is that expected?

I think so, yes.

> The manual section 'Returning and Accepting Multiple Values' does
> mention tail position, but only in passing, and it isn't obvious to me
> why that restriction should exist.

'or' needs to test whether the results of the non-final expressions are
true or false.  It's not clear how this should be generalized to
multiple values.

 Mark

Re: GNU Guile 2.0.10 released

[Mark H Weaver]

Eli Zaretskii  writes:
> Mark told me that someone succeeded in building MinGW Guile with
> pthreads, and he built pthreads himself.

Someone reported having done so, but that turned out to be false.  At
this point, I'm not aware of anyone who has built a working threaded
Guile for MinGW.

 Mark

Re: unicode support

[Mark H Weaver]

Игорь Гайсин  writes:

> Hi, dmitriy. Try to add this line to head your scripts
>
> (setlocale LC_ALL "")
> (set-port-encoding! (current-input-port) "utf-8")
> (set-port-encoding! (current-output-port) "utf-8")

The 'setlocale' call is indeed important.

The next two lines shouldn't be needed, and would cause breakage on
non-utf8 locales.

 Regards,
   Mark


> Dima  writes:
>
>> Hi.
>>Did something change from guile 1.8.x to 2.0 with regards to unicode 
>> support.
>>
>> I have a script that has strings written on russian without much care, and 
>> it does not work on 2.0.
>>
>> I want to make sure whether there are changes in guile or something else.
>>
>> Thanks, regards,
>>   Dima.

Re: Does Guile have a thread limit?

[Mark H Weaver]

"Diogo F. S. Ramos"  writes:

> The following program is aborted:
>
> (define number-of-thread 1000)
>
> (do ((i number-of-thread (- i 1)))
> ((zero? i))
>   (call-with-new-thread (lambda () (sleep 42

I looked into this, and the issue is that for every thread that's ever
put into Guile mode, Guile creates a pipe which is used to wake it up
while it's sleeping or waiting in 'select', e.g. if an async is queued.

Therefore, two file descriptors are allocated for each such thread.

Unfortunately, if you run out of file descriptors while creating a
thread, it simply aborts, which is obviously suboptimal.  The relevant
code is in threads.c:563.

Mark

Re: Does Guile have a thread limit?

[Mark H Weaver]

Hi,

Taylan Ulrich Bayirli/Kammer  writes:

> taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes:
>
>> While C code using up many FDs and then calling scm_with_guile() might
>> abort --which should be a rare case--, `call-with-new-thread' neatly
>> raises an exception ... almost.  In the REPL, when I repeatedly enter
>> our test-case snippet of launching 1k sleeping threads, I manage to
>> make Guile hang.  In case anyone's interested, here's the backtraces
>> of all threads when I attach with GDB: http://sprunge.us/VZMY
>
> I solved this now, the problem was that I didn't think about cleanup at
> all after a failed thread initialization; though in this case the only
> thing that needed to be done was to call GC_unregister_my_thread.
>
> Does anyone have time to review this patch?  It passes `make check' and
> neatly throws an exception when a thread fails to initialize due to
> running out of FDs.

One serious problem with this patch is that it changes the API (and ABI)
of 'scm_init_guile'.  This function has never returned an error code and
thus has always been assumed to succeed.  This patch would allow it to
fail and return an error code that no one ever checks.  There are other
problems with the patch, but I'm not sure it's worth enumerating them
because I think this is the wrong approach.

The fundamental problem here is that currently, programs must be careful
to limit the number of threads that have ever been put into Guile mode,
just as they must limit the number of open files.  If they do that, then
they won't ever encounter this error.  If they run out of FDs then they
are screwed, and this patch doesn't change that fact, it merely trades
one kind of ungraceful exit for another.

A proper solution would eliminate that limitation.  Once we've done
that, then there will no longer be a need for these new APIs that you
would introduce, nor for the error-prone code paths that they would
entail.

> (Well, in the REPL, one gets flooded with readline errors when that
> fails to acquire an FD because the succeeding threads ate up all the
> available ones, but that's a different issue.)

I don't think it's a different issue.  Rather, I think it demonstrates
that this patch doesn't really solve the problem in practice.

  Mark

Re: #:getter procedure returns unexpected value in GOOPS

[Mark H Weaver]

"Diogo F. S. Ramos"  writes:

> When using GOOPS, if a class has a second slot, the #:getter procedure
> of the first slot returns the value of the second slot when applied to
> an instance of a subclass.
>
> (use-modules (oop goops))
>
> (define-class  ()
>   (a #:init-form 'foo #:getter foo-a)
>   (b #:init-form 42))
>
> (define-class  ()
>   (a #:init-form 'bar))
>
>   (foo-a (make )) => foo
>   (foo-a (make )) => 42
>
> I expected:
>
>   (foo-a (make )) => bar

Indeed, CLOS behaves as you expected, and GOOPS should probably behave
the same way.  However, it appears that overriding the attributes of
slots in subclasses has not worked this way in a long time, if ever.

I tried this example on both Guile 1.8 and Guile 1.6, and neither of
them behave as you expected.  Instead they complain that there's no
applicable method for 'foo-a'.

Can you please send a bug report to bug-gu...@gnu.org?

For now, I suggest adding an initialize method for , like this:

--8<---cut here---start->8---
(use-modules (oop goops))

(define-class  ()
  (a #:init-form 'foo #:getter foo-a)
  (b #:init-form 42))

(define-class  ())

(define-method (initialize (obj ) args)
  (slot-set! obj 'a 'bar)
  (next-method))

(foo-a (make )) => foo
(foo-a (make )) => bar
--8<---cut here---end--->8---

 Regards,
   Mark

Re: What is the output port of `system*'?

[Mark H Weaver]

"Diogo F. S. Ramos"  writes:

>> "Diogo F. S. Ramos"  writes:
>>
>>> The following program doesn't output to a string, which I expected.
>>>
>>> (define foo
>>>   (with-output-to-string
>>> (lambda ()
>>>   (system* "ls" "/tmp"
>>
>> As the manual says about `system*':
>>> The command is executed using fork and execlp.
>>
>> That implies certain restrictions.  See (ice-9 popen), documented in
>> (info "(guile) Pipes").  That also spawns a process but sets its stdout
>> to a Unix pipe for which it gives you a Guile port, from which you can
>> drain the output to construct a string.
>
> Thanks for the pipe reference.
>
> Indeed, something can probably be written with these pipes to achieve
> what my sample program tried to do tho the higher issue remains.
>
>> (Details: `with-output-to-string' could be said to work by setting the
>> current output port to some special value (a "string port" I guess) that
>> tells the deepest Guile IO procedures to construct a string instead of
>> writing to a file descriptor; when you spawn a separate process you
>> obviously lose this ability, the spawned process simply inherits the
>> current stdout file descriptor of the Guile process and writes to that.)
>
> I'm not sure about the underlying implementation, but I expected all
> output to (current-output-port) going to the resulting string, hence my
> question.

All output to (current-output-port) _does_ go into the resulting string.
However, when you launch a subprocess using 'system*', the output of
that process does _not_ go to (current-output-port).  Instead, it goes
to POSIX file descriptor 1, which is left unchanged by the Scheme
constructs that change (current-output-port).  Ditto for
(current-input-port) and (current-error-port).  There are at least two
reasons for this:

* POSIX file descriptors cannot represent all of the types of ports
  supported by Guile, such as string ports and custom binary ports.

* The current-output-port is a fluid variable, which among other things
  is thread-local.  Obviously, POSIX file descriptors are per-process.

> As an example of this behavior, I point to the following Racket program:
>
> #lang racket
>
> (define foo
>   (with-output-to-string
> (lambda ()
>   (system* (find-executable-path "ls") "/tmp"

I agree that it would be nice to make this work as you expected.  In
order to handle ports that are not simply raw POSIX file descriptors
(e.g. string ports), we'd need to create a pipe behind the scenes, and
arrange for thread(s) in the current process to copy data between the
pipe(s) and the Scheme port.

I'd like to do this at some point, although for the sake of backward
compatibility we'd probably have to leave 'system' and 'system*' as-is
and create a new API that works more intuitively.

  Mark

Re: #:getter procedure returns unexpected value in GOOPS

[Mark H Weaver]

David Pirotte  writes:

> Hi Mark,
>
>> > When using GOOPS, if a class has a second slot, the #:getter procedure
>> > of the first slot returns the value of the second slot when applied to
>> > an instance of a subclass.
>> >
>> > (use-modules (oop goops))
>> >
>> > (define-class  ()
>> >   (a #:init-form 'foo #:getter foo-a)
>> >   (b #:init-form 42))
>> >
>> > (define-class  ()
>> >   (a #:init-form 'bar))
>> >
>> >   (foo-a (make )) => foo
>> >   (foo-a (make )) => 42
>> >
>> > I expected:
>> >
>> >   (foo-a (make )) => bar
>
>> Indeed, CLOS behaves as you expected...
>
> Are you sure about that?

Read this:

  http://clhs.lisp.se/Body/07_ec.htm

In particular:

 "In general, more than one class among C and its superclasses can
  define a slot with a given name.  In such cases, only one slot with
  the given name is accessible in an instance of C, and the
  characteristics of that slot are a combination of the several slot
  specifiers, computed as follows: [...]"

Here's a transcript of an SBCL session I tried on Debian:

--8<---cut here---start->8---
This is SBCL 1.0.57.0.debian, an implementation of ANSI Common Lisp.
More information about SBCL is available at .

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
* (defclass foo ()
   ((a :initform 'foo :reader foo-a)
(b :initform 42)))

#
* (defclass bar (foo)
   ((a :initform 'bar)))

#
* (values (foo-a (make-instance 'foo)) (foo-a (make-instance 'bar)))

FOO
BAR
--8<---cut here---end--->8---

Unknown_lamer on #guile tried the same thing with a different
implementation of Common Lisp, and got the same answer.

 Mark

Re: Welcome Mark as new co-maintainer

[Mark H Weaver]

l...@gnu.org (Ludovic Courtès) writes:
> I’m happy to announce that Mark H. Weaver has now been officially
> appointed as Guile co-maintainer by the GNU Project overseers, along
> with Andy and myself.

Thank you all for the kind words and warm welcome.  I believe that Guile
is a tremendously important project, not so much because of where it is
today, but because of its future potential and its special role in the
GNU project as a whole.  It's also a lot of fun to work on, not least of
all because of the talented and friendly community of hackers around it.

Although I had a few decades of experience before getting involved here,
I sincerely feel that I've learned a great deal from this community,
both technically and socially.  I'm grateful to Andy and Ludovic for
inviting me into this role of co-maintainer, a role for which both of
them are exceptionally well suited, whereas I still have much to learn.

Anyway, it's been a fun ride, and I think there are exciting times ahead
for Guile.  Thank you all for being a part of that :)

 Mark

Re: Macro expansion: unknown location

[Mark H Weaver]

l...@gnu.org (Ludovic Courtès) writes:

> A workaround is to get location info from the outer syntax object ‘x’:
>
> (define-syntax zero!
>   (lambda (x)
> (syntax-case x ()
>   ((_ var)
>(begin
>(unless (identifier? #'var)
>  (syntax-violation 'zero! "identifier expected" x)) ; <-- here
>#'(set! var 0))
>
> I’m not sure why ‘var’ has no location info.

Currently, Guile stores location info using a weak-key hash table, keyed
on the datums read from the file.  This means that location info cannot
be stored for bare symbols or other immediate values (booleans,
characters, small exact integers).

I intend to fix this at some point, but it entails using a reader with a
different API than the standard reader.  The new reader would have to
return symbols and immediates wrapped in some other data type.

  Mark

Re: Macro expansion: unknown location

[Mark H Weaver]

Mark H Weaver  writes:

> Currently, Guile stores location info using a weak-key hash table, keyed
> on the datums read from the file.  This means that location info cannot
> be stored for bare symbols or other immediate values (booleans,
> characters, small exact integers).

Symbols are not immediate values, so I shouldn't have written "other".

However, because they are interned, they behave like immediates in the
following sense: 'read' returns the same (by 'eq?') object for any two
occurrences of a given symbol.  Therefore, there's no way to associate
source location information with each copy of the same symbol.

 Mark

Re: Compiling for Android

[Mark H Weaver]

b...@beuc.net writes:

> Check ./docs/text/system/libc/OVERVIEW.text from the Android NDK:
[...]
> pthread_once():
[...]
>   The current implementation of pthread_once() lacks the necessary support of
>   multi-core-safe double-checked-locking (read and write barriers).

This is the only bit that worries me.  It sounds like Bionic's
pthread_once() might not work properly on multi-core systems.  We rely
on 'pthread_once' for thread-safe lazy initialization in libguile.

 Mark

Re: cannot compile: srfi-10 define-reader-ctor 'hash '#,(

[Mark H Weaver]

Jan Nieuwenhuizen  writes:
> I mean an standardized, ascii/utf-8 non-opaque (#)
> representation of hash tables, something like
>
> #,(hash (key0 value0) .. (keyn valuen))
>
> that upon `read', produces a hash table.

This has been proposed several times before, and although it generally
sounds like a nice idea, there are unfortunately several complications:

1. There are at least three different kinds of hash tables in Guile:
   native legacy Guile hash tables, SRFI-69 hash tables, and R6RS hash
   tables.

2. For each of these three kinds of hash tables, they are further
   divided into multiple flavors depending on the equality predicate and
   associated hash function: eq?, eqv?, equal?, and potentially other
   kinds defined by the user.

3. If the equality predicate is eq? or eqv?, then there's no way to
   write a hash table and then read it back in without losing
   information.  For both of these kinds of hash tables, mutable objects
   that produce the same output can either be the same object or
   different objects.

4. Unlike SRFI-69 and R6RS hash tables, native legacy Guile hash tables
   do not keep a record of which equality predicate is used to store
   their elements.  Instead, it is the user's responsibility to use the
   correct accessors (hash-ref, hashq-ref, hashv-ref, hashx-ref)
   mutators, and other procedures.  It is even possible to use both
   hashq-set! and hashv-set! on the same hash table, although it's
   almost certainly a bad idea to do so.  This means that when asked to
   print a native hash table, Guile doesn't have the needed information
   to print what equality predicate the hash table uses.

I should also mention that it would not be enough to allow 'read' to
read hash tables.  To compile a source file containing literal hash
tables, we'd also need to add support to our assembler and loader to
serialize hash tables to .go files and load them back in.

Regarding complication #1, at some point I'd like to at least merge
SRFI-69 and R6RS hash tables into the same underlying data type.  How to
merge those with native Guile hash tables is not obvious because of
complication #4.  One idea is to record the equality predicate in the
hash table, but allow the predicate to be "not yet determined" when a
hash table is created by 'make-hash-table' before its first element is
added.

If that problem was solved, then complication #2 could be handled by
annotating the external representation with the equality predicate.

I see no good solution to complication #3, but I suppose we could
document that information can be lost.

  Mark

Re: guile-2.0.11 installation on system with 2.0.5

[Mark H Weaver]

Neil Jerram  writes:

> On 2014-09-15 02:16, m...@netris.org wrote:
>> Federico Beffa  writes:
>>
>>> I notice that there is an /etc/ld.so.cache file. Do I somehow need
>>> to update
>>> it?
>>
>> Yes, you update it by running "ldconfig" as root.  This needs to be
>> done
>> when installing libraries outside of your package manager, not only for
>> Guile, but for essentially all packages containing libraries.
>
> I agree that this will allow /usr/local/bin/guile to load
> /usr/local/lib/libguile*.
>
> However, won't it also cause /usr/bin/guile (2.0.5) to load
> /usr/local/lib/libguile* (2.0.11) ?

True, but I don't see why anyone would want to run 2.0.5 anymore.  It
has lots of bugs and other problems that are fixed in 2.0.11, so that's
probably a good thing :)

> If not, what is the mechanism
> that tells /usr/bin/guile (2.0.5) to load /usr/lib/libguile* (2.0.5)
> instead of /usr/local/lib/libguile* (2.0.11) ?

This is not a guile-specific issue.  Any time you want to use two
different versions of the same shared library, you need to rely on some
other mechanism.

If you really want to preserve your ability to run the old buggy 2.0.5
libguile, then first of all you need to avoid installing 2.0.11 in a
directory that's in /etc/ld.so.conf (or /etc/ld.so.conf.d), because even
if you avoid running "ldconfig" yourself, your package manager will run
it the next time you install a library package.  So, you'd have to
install 2.0.11 to a different prefix.

Then you need a way for your new PREFIX/guile to find its shared library
in a non-standard place, and for that you'd best use rpaths.  See
, but one way is to
make sure that LD_RUN_PATH is set to PREFIX/lib at the time that the
'guile' executable is linked.

Note that in GNU Guix , rpaths are always used
and this issue wouldn't have arisen.

 Regards,
   Mark

Re: guile-2.0.11 installation on system with 2.0.5

[Mark H Weaver]

Chris Vine  writes:
> But quite honestly, if you want to experiment with newer software don't
> use an archaic distribution such as debian stable, which is out of date
> even when first released.

Umm, I think it's quite reasonable to run a distribution of older
software that has gone through a long testing period and works well
together, and then to compile more recent versions of selected packages
of particular interest.

In my experience, every time I've run a more cutting-edge distro, I
inevitably end up hitting bugs in software I don't care about when I'm
just trying to get something else done, leaving me less time to work on
the packages I choose to focus on.  YMMV, of course.

 Mark

Re: emacs interaction

[Mark H Weaver]

Matt Wette  writes:

> On Sep 20, 2014, at 12:30 AM, Thien-Thi Nguyen  wrote:
>> It would help to post an example (w/ tabs converted to spaces)
>> showing what you desire.  That way we have a clear goal.
>
> I get
> (define-module (mymod)
>   #:export (blablabla foo bar
>  a b c)
>   #:export-syntax (blablablam foom barm
>am bm cm)
>   )
>
> I want 
> (define-module (mymod)
>   #:export (blablabla foo bar
>  a b c)
>   #:export-syntax (blablablam foom barm
>  am bm cm)
>   )

I'm fairly sure we're not seeing the same thing you're seeing in this
email.  It looks as though perhaps you've written the email with a
variable-width font, and used enough spaces on the continuation lines to
make things line up properly on your display, whereas many of us are
viewing this email with a fixed-width font, and nothing lines up at all.

The following web page might show you what your email looks like to us:

http://lists.gnu.org/archive/html/guile-user/2014-09/msg00093.html

 Regards,
   Mark

Re: smob mark functions in 2.0

[Mark H Weaver]

Hi Andy,

Reviving a 3-year-old thread...

Andy Wingo  wrote in November 2011:

> If you do implement a SMOB marking function, and you touch Scheme
> objects in that marking function, you need to be very careful.
> 
> Specifically, there is a warning in gc/gc_mark.h:
> 
> /* WARNING: Such a mark procedure may be invoked on an unused object*/
> /* residing on a free list.  Such objects are cleared, except for a */
> /* free list link field in the first word.  Thus mark procedures may*/
> /* not count on the presence of a type descriptor, and must handle this */
> /* case correctly somehow.  */
> 
> So, your mark function might see freed objects.  This is terrible, but
> it is the way that it is.  The key is that, if you touch a Scheme object
> in your mark function, to first do a check on that object, to see that
> it is valid.  You can check the TC bits of the first word, or otherwise
> check that other words are non-NULL.

Andy Wingo  later replied to Ludovic:

> On Thu 24 Nov 2011 00:12, l...@gnu.org (Ludovic Courtès) writes:
>
>>> So, your mark function might see freed objects.  This is terrible, but
>>> it is the way that it is.  The key is that, if you touch a Scheme object
>>> in your mark function, to first do a check on that object, to see that
>>> it is valid.  You can check the TC bits of the first word, or otherwise
>>> check that other words are non-NULL.
>>
>> What about making that check in libguile before invoking the user’s mark
>> function?
>
> Yes, we do that.  I think you wrote that code!  The problem was in a
> mark function, accessing *other* Scheme objects.

Mark H Weaver  writes:

> Andy Wingo  writes:
>> Specifically, there is a warning in gc/gc_mark.h:
>>
>> /* WARNING: Such a mark procedure may be invoked on an unused object
>> */
>> /* residing on a free list.  Such objects are cleared, except for a 
>> */
>> /* free list link field in the first word.  Thus mark procedures may
>> */
>> /* not count on the presence of a type descriptor, and must handle this 
>> */
>> /* case correctly somehow.  
>> */
>>
>> So, your mark function might see freed objects.
>
> How can this happen?  If you are marking an object, then presumably it
> is still reachable, and therefore the objects it references are also
> still reachable.  If any of those reachable objects has been freed,
> isn't that already a bug of a different kind?  What am I missing?

I never received an answer to this question.  At the time it was merely
a curiosity, but now I have a more pressing need to understand what's
going on here.

Regards,
  Mark

Re: Guile 2.0 interpretation vs compilation

[Mark H Weaver]

Marko Rauhamaa  writes:

> Consider this program:
>
> ===begin test.scm===
> (define (hello) #f)
> (format #t "~S\n" (procedure-name hello))
>
> (define (xyz)
>   (define (hello) #f)
>   (format #t "~S\n" (procedure-name hello)))
>
> (xyz)
> ===end test.scm=
>
> If I run:
>
>$ guile --no-auto-compile test.scm
>hello
>#f
>$ guile --no-auto-compile test.scm
>hello
>#f

This is expected but not ideal.  Our primitive evaluator does not
preserve non-toplevel variable names, and therefore the associated
procedure names are lost.  Maybe we can fix this in Guile 2.2.

>$ guile test.scm
>;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
>;;;   or pass the --no-auto-compile argument to disable.
>[...]
>hello
>hello
>$ guile test.scm
>hello
>hello
>$ guile --no-auto-compile test.scm
>hello
>hello

The reason this last one works is because the .go file was created in
the preceding compiled runs, and --no-auto-compile does not inhibit the
use of pre-existing .go files (unless the .scm file is newer, of
course).

 Regards,
   Mark

Re: Guile 2.0 interpretation vs compilation

[Mark H Weaver]

Marko Rauhamaa  writes:

> Mark H Weaver :
>
>> This is expected but not ideal. Our primitive evaluator does not
>> preserve non-toplevel variable names, and therefore the associated
>> procedure names are lost. Maybe we can fix this in Guile 2.2.
>
> I noticed the issue because I have software that depends on the
> procedure-name of inner functions. I can live with this guile-2.0 quirk
> if it is considered a guile defect.

I consider it a guile defect, yes.  Can you please send a bug report to
bug-gu...@gnu.org?  I'll look into whether it can be fixed in 2.0.x, and
try to come up with a workaround.

 Thanks,
   Mark

Re: Guile 2.0 interpretation vs compilation

[Mark H Weaver]

Marko Rauhamaa  writes:

> Mark H Weaver :
>
>> This is expected but not ideal. Our primitive evaluator does not
>> preserve non-toplevel variable names, and therefore the associated
>> procedure names are lost. Maybe we can fix this in Guile 2.2.
>
> I noticed the issue because I have software that depends on the
> procedure-name of inner functions. I can live with this guile-2.0 quirk
> if it is considered a guile defect.

Here's a simple workaround for now, though unfortunately it will mostly
prevent our compiler from doing optimizations on code that uses this
'define' macro:

  (define-syntax define
(syntax-rules ()
  ((define (proc . args) body body* ...)
   (define proc
 (let ((p (lambda args body body* ...)))
   (set-procedure-property! p 'name 'proc)
   p)))
  ((define name expr)
   ((@ (guile) define) name expr

Does this work for you?

Alternatively, you could make sure to always compile your code, in which
case 'procedure-name' should work properly.

  Mark

Re: Guile: passing a file descriptor

[Mark H Weaver]

Marko Rauhamaa  writes:

> A PF_UNIX socket can be used to pass an open file descriptor between
> processes. However, the relevant control message seems to be missing in
> Guile's POSIX facilities (send only accepts a data message).

Indeed, we lack that functionality in core Guile.

> Am I mistaken, and are there any plans to add control message support to
> Guile's library?

I'm not aware of any plans to add it, but if we can agree on an API and
someone contributed a suitable implementation, I don't see why we
couldn't add it.

In the meantime, it's easy to extend Guile with your own bits of C code.

  http://www.gnu.org/software/guile/manual/html_node/C-Extensions.html

You could also probably use our Dynamic FFI to do the job entirely from
Scheme, although we don't have any automatic way of dealing with C
preprocessor macros, so they would probably have to be hard-coded.

  
http://www.gnu.org/software/guile/manual/html_node/Foreign-Function-Interface.html

 Mark

Re: iconv or something like that

[Mark H Weaver]

Konrad Makowski  writes:
> Is there any solution to convert charset from one encoding to another?

Yes, but character encodings are only relevant when converting between a
sequence of _bytes_ (a bytevector), and a sequence of _characters_ [*]
(a string).  These conversions happen implicitly while performing I/O,
converting Scheme strings to/from C, etc.

[*] More precisely, Scheme strings are sequences of unicode code points.

It doesn't make sense to talk about the encoding of a Scheme string, or
to convert a Scheme string from one encoding to another, because they
are not byte sequences.

It sounds like you already have a Scheme string that was incorrectly
decoded from bytes, and are asking how to fix it up.  Unfortunately,
this won't work, because many valid ISO-8859-2 byte sequences are not
valid UTF-8, and will therefore lead to decoding errors.

> I have database in iso-8859-2 but my script runs in utf-8. I use dbi module.

Having looked at the guile-dbi source code, I see that it always uses
the current locale encoding when talking to databases.  Specifically, it
always uses 'scm_from_locale_string' and 'scm_to_locale_string'.  For
your purposes, you'd like it to use 'scm_from_stringn' and
'scm_to_stringn' instead, with "ISO-8859-2" as the 'encoding' argument.

My knowledge of modern databases is limited, so I'm not sure how this
problem is normally dealt with.  It seems to me that, ideally, strings
in databases should be sequences of Unicode code points, rather than
sequences of bytes.  If that were the case, then this problem wouldn't
arise.

It would be good if someone with more knowledge of databases would chime
in here.

In the meantime, I can see a few possible solutions/workarounds:

* Enhance guile-dbi to include an 'encoding' field to its database
  handles, add a new API procedure to set it, and use it in all the
  appropriate places.  This only makes sense if database strings are
  conceptually byte sequences, otherwise it should probably be fixed in
  some other way.

* Hack your local copy of guile-dbi to use 'scm_from_stringn' and
  'scm_to_stringn' with a hard-coded "ISO-8859-2" in the appropriate
  places.

* Use 'setlocale' to set a ISO-8859-2 locale temporarily while
  performing database queries.

Which database are you using?

 Mark

Re: guile - sending emails

[Mark H Weaver]

Konrad Makowski  writes:

> I use msmtp for my needs.
>
> (define (send-email)
>  (let ((port (open-pipe* OPEN_BOTH "msmtp" SMTP_MAIL_TO_SECRETARY)))
>  (display (smtp-body) port)
>  (close-pipe port)))

You should use OPEN_WRITE instead of OPEN_BOTH here, but otherwise it
looks good to me.

 Mark

Re: goops and equal?

[Mark H Weaver]

Jan Nieuwenhuizen  writes:

> In the example below, my implementation of equal? is only being 
> used if both types are the same.  That surprised me; the manual
> says equal? becomes a generic and for normal generics this works
> as I expect.

'equal?' is actually a "primitive generic", which means that it's a core
C function that handles the cases it knows about, and only calls out to
user-defined GOOPS methods for cases that aren't handled internally.

One of the facts known by core 'equal?' is that objects of different
types are not equal.

It may be that we should relax this somewhat, but IMO it's probably a
bad idea to allow instances of your user-defined type to be 'equal?' to
symbols.  Out of curiosity, why do you want this?

BTW, in order to allow user-defined types to be 'equal?' to built-in
types, we'd also need to change our hashing function for 'equal?'-based
hash tables.

  Regards,
Mark