bug#12827: [PATCH] Tweak web modules, support relative URIs
Hi Daniel, Daniel Hartwig writes: > * Terminology > > The terminology used in latest URI spec. (RFC 3986) is not widely used > elsewhere. Not by Guile, not by the HTTP spec., or other sources. > Specifically, it defines these terms: > > - URI: scheme rest ... [fragment] > - Absolute-URI: scheme rest ... [fragment] > - Relative-Ref: rest ... [fragment] > - URI-Reference: Absolute-URI | Relative-Ref > > where as HTTP and other sources use the terms from an earlier URI > spec. (RFC 2396): > > - Absolute-URI: scheme rest ... [fragment] > - Relative-URI: rest ... [fragment] > - URI, URI-Reference: Absolute-URI | Relative-URI > > With this patch I have opted to use the later, more common terms. > This has the advantage of not requiring massive renaming or > duplicating of most procedures to include, e.g. > ‘uri-reference-scheme’, as we just use the term ‘uri’ to refer to > either type. > > If this is undesired, it can easily be reworked to use the terminology > from RFC 3986. Thanks for your careful work on this, and especially for calling our attention to the terminology changes introduced in the latest URI spec. My preference would be to use the newer RFC 3986 terms. To my mind, the key question is: which type (Absolute-URI or URI-Reference) is more commonly appropriate in user code, and thus more deserving of the short term "URI". I would argue that Absolute-URIs are more often appropriate in typical user code. The reason is that outside of URI-handling libraries, most code that deals with URIs simply use them as universal pointers, i.e. they implicitly assume that each URI is sufficient by itself to identify any resource in universe. Working with URI-References is inherently trickier and more error-prone, because code that handles them must do some additional bookkeeping to associate each URI-Reference with its _context_. It is inconvenient to mix URI-References from different contexts, and they must be converted when moved from one context to another. For typical code, the simplest and safest strategy for dealing with URI-References is to convert them to Absolute-URIs as early as possible, preferably as the document is being read. (Of course, there are special cases such as editors where it is important to preserve the URI-References, but that is not the typical case). Therefore, I think that Absolute-URI is more deserving of the short term "URI", and furthermore that existing code outside of (web uri) that refers to URIs should, by default, be assumed to be talking about Absolute-URIs. Only after some thought about whether a procedure handles relative references properly should its type be changed to accept URI-References. > * API compatability > > Presently, all APIs work only with absolute URIs. You can not use > string->uri or build-uri to produce any relative URIs, neither are > other procedures (generally) expected to work correctly if given them. For the reasons given above, I think that it is a virtue, not a flaw, i.e. I think that latest URI spec (RFC 3986) got this right. It is important to clearly distinguish Absolute-URIs from URI-References. Despite their overlapping syntax, they are very different concepts, and must not be conflated. Here's what I suggest: instead of extending 'string->uri' and 'build-uri' to produce relative URIs, rename those extended procedures 'string->uri-reference' and 'build-uri-reference'. These are long names, but that's okay because users should think twice before using them, and that's seldom needed. Then, we extend 'string->uri' and 'build-uri' in a different way: we extend them to handle relative references in their *inputs*, but continue to provide absolute *outputs*, by adding an optional keyword argument '#:base-uri'. This would make it easy to implement the simplest and safest strategy outlined above with a minimum of code changes. What do you think? Mark
bug#10474: Building guile 2.x under mingw + msys
On Tue 19 Feb 2013 18:53, Eli Zaretskii writes: >> + (define (unc-path?) >> + ;; Universal Naming Convention (UNC) paths start with \\, and >> + ;; are always absolute. >> + (string-prefix? "" path)) > > A UNC file name can also begin with 2 slashes, as in "//foo/bar/". In > general, Windows system calls treat both kinds of slashes identically. I've fixed this, I think. Thanks for the note, Andy -- http://wingolog.org/
bug#12827: [PATCH] Tweak web modules, support relative URIs
On 24 February 2013 18:45, Mark H Weaver wrote: > Hi Daniel, > > Daniel Hartwig writes: >> * Terminology >> >> The terminology used in latest URI spec. (RFC 3986) is not widely used >> elsewhere. Not by Guile, not by the HTTP spec., or other sources. >> Specifically, it defines these terms: >> >> - URI: scheme rest ... [fragment] >> - Absolute-URI: scheme rest ... [fragment] >> - Relative-Ref: rest ... [fragment] >> - URI-Reference: Absolute-URI | Relative-Ref >> >> where as HTTP and other sources use the terms from an earlier URI >> spec. (RFC 2396): >> >> - Absolute-URI: scheme rest ... [fragment] >> - Relative-URI: rest ... [fragment] >> - URI, URI-Reference: Absolute-URI | Relative-URI >> > My preference would be to use the newer RFC 3986 terms. To my mind, the > key question is: which type (Absolute-URI or URI-Reference) is more > commonly appropriate in user code, and thus more deserving of the short > term "URI". > > I would argue that Absolute-URIs are more often appropriate in typical > user code. The reason is that outside of URI-handling libraries, most > code that deals with URIs simply use them as universal pointers, > i.e. they implicitly assume that each URI is sufficient by itself to > identify any resource in universe. Right. RFC 3986 makes a convincing argument for the new terminology. These notes about usage also reflect the sentiment in that document. FWIW, I sat mostly on the fence, finally going away from URI-Reference due to these concerns I expressed in an earlier email: > The API seems less clean, and it is not immediately clear > that uri? is not the top of the URI-like type hierarchy. The other > functions only indicate “uri” in their name. I did not > wish to introduce parallel “build-uri-reference”, etc. for each of > these, and did consider adding #:reference? on some to select > weaker validation. and looking at some other Scheme URI modules. However, having read over your comments I think that we could reasonably get away with just introducing the procedures you mention below and not bother about renaming (or duplicating) the field getters to ‘uri-reference-path’ etc.. > Here's what I suggest: instead of extending 'string->uri' and > 'build-uri' to produce relative URIs, rename those extended procedures > 'string->uri-reference' and 'build-uri-reference'. These are long > names, but that's okay because users should think twice before using > them, and that's seldom needed. In your proposed solution, ‘uri?’ and ‘uri-reference?’ are the predicates and they respond according to the RFC rather than internal Guile details? That is: (uri? (string->uri-reference "http://example.net/";)) => #t (uri-reference? (string->uri-reference "http://example.net/";)) => #t (uri? (string->uri-reference "foo")) => #f or …? > Then, we extend 'string->uri' and 'build-uri' in a different way: we > extend them to handle relative references in their *inputs*, but > continue to provide absolute *outputs*, by adding an optional keyword > argument '#:base-uri'. This would make it easy to implement the > simplest and safest strategy outlined above with a minimum of code > changes. This strategy does reflect the recommendation of RFC 3986 to resolve the references as they are read. Also an elegant API, as it encourages immedately resolving uri-references and never creating (or considering to create) the context-sensitive relative-refs. > > What do you think? > I quite like it, particularly the last part about #:base-uri. Ludo, I think this is basically what you were suggesting in the first place? :-) .
bug#10474: Building guile 2.x under mingw + msys
On Tue 19 Feb 2013 19:00, Eli Zaretskii writes: >> >* libguile/load.c (scm_init_load_path) [__MINGW32__]: Convert >> >backslashes to forward slashes in values of GUILE_LOAD_PATH and >> >GUILE_LOAD_COMPILED_PATH. >> >> Is this necessary now that backslash is recognized as a file name >> separator? > > Possibly. But are you sure there's no more references to '/' alone in > C code (as opposed to in Scheme)? I am pretty sure, yes. I just pushed some patches to make sure this is the case, altering load.c's logic to be more structurally similar to boot-9.scm's and preferring backslashes on Windows. So with these last commits, hopefully native MinGW builds are supported. Would you mind testing again? Please send a new mail to bug-guile@gnu.org to track any new failures. Thanks, Andy -- http://wingolog.org/
bug#10621: Incorrect usage of hash procedures in (ice-9 mapping)
On Wed 28 Nov 2012 16:56, Daniel Hartwig writes: > A short module, it is not hard to fix, though given all of the above it > makes sense to simply remove it. I think removing it is the right thing. Even with your patch it still won't work because the default "table constructor" is a vector instead of a hash table. Regardless of whether it were new or old, I would much prefer to remove this module than to document it properly, especially given the existence of alternatives like srfi-69 and rnrs hash tables as you mention. I will apply your patch (so that the compile warnings go away), add a deprecation notice, and remove the module in master. Andy -- http://wingolog.org/
bug#10622: Bugs in decompile-assembly.scm
On Fri 27 Jan 2012 09:26, Mark H Weaver writes: > These are genuine bugs: > > language/glil/decompile-assembly.scm:174:21: warning: possibly unbound > variable `make-glil-local' > language/glil/decompile-assembly.scm:170:21: warning: possibly unbound > variable `make-glil-local' Fixed by removing this module. Besides not working in the sense of "fails to compile", it was only ever about 5% complete. Andy -- http://wingolog.org/
bug#10474: Building guile 2.x under mingw + msys
> From: Andy Wingo > Cc: l...@gnu.org, 10474-d...@debbugs.gnu.org > Date: Sun, 24 Feb 2013 14:25:06 +0100 > > So with these last commits, hopefully native MinGW builds are supported. > Would you mind testing again? Thanks, will do when I have time.
bug#10627: char-ready? is broken for multibyte encodings
On Sat 28 Jan 2012 11:21, Mark H Weaver writes: > The R5RS specifies that if 'char-ready?' returns #t, then the next > 'read-char' operation is guaranteed not to hang. This is not currently > the case for ports using a multibyte encoding. > > 'char-ready?' currently returns #t whenever at least one _byte_ is > available. This is not correct in general. It should return #t only if > there is a complete _character_ available. This procedure is omitted in the R6RS because it is not a good interface. Besides its semantic difficulties, can you think of a sane implementation for multibyte characters? I suggest we document that this procedure only works correctly in encodings with 1-byte characters and recommend that people use u8-ready? instead. Andy -- http://wingolog.org/
bug#13768: --without-posix code uses scm_getpid() in libguile-2.0.2
On Wed 20 Feb 2013 00:38, Jan Schukat writes: > What happens is, in random.c in random_state_of_last_resort on line 668 > scm_getpid is used to seed the random generator. So either a > preprocessor switch or a hand constructed scm like in scm_getpid > (scm_from_ulong(getpid())) should be used there. Fixed, thanks for the report. Andy -- http://wingolog.org/
bug#10645: Guile 2.0.3: fails make check on s390x in test-ffi
On Tue 31 Jan 2012 19:47, Mark H Weaver writes: > Rob Browning writes: >> Make check fails on the s390x architecture: > [...] >> ERROR: incorrect result (7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 >> 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 >> 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 >> 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7) >> FAIL: test-ffi >> >> You can see the full build log here: >> >> https://buildd.debian.org/status/fetch.php?pkg=guile-2.0&arch=s390x&ver=2.0.3%2B1-3&stamp=1327875038 > > I suspect this is a bug in libffi. The most recent Debian build log for > libffi on s390x/sid shows several failures during its 'make check'. Given that Guile 2.0.5 is in Debian and working fine on s390x, I am closing this bug. Andy -- http://wingolog.org/
bug#10646: Guile 2.0.3 fails make check on armhf in test-pthread-create
Hi,
On Mon 30 Jan 2012 04:32, Rob Browning writes:
> Make check fails on the armhf architecture:
>
> /bin/bash: line 5: 31270 Segmentation fault srcdir="." builddir="."
> GUILE_AUTO_COMPILE=0 "../../meta/uninstalled-env" ${dir}$tst
> FAIL: test-with-guile-module
> PASS: test-scm-with-guile
> /bin/bash: line 5: 31333 Segmentation fault srcdir="." builddir="."
> GUILE_AUTO_COMPILE=0 "../../meta/uninstalled-env" ${dir}$tst
> FAIL: test-scm-spawn-thread
> /bin/bash: line 5: 31366 Segmentation fault srcdir="." builddir="."
> GUILE_AUTO_COMPILE=0 "../../meta/uninstalled-env" ${dir}$tst
> FAIL: test-pthread-create
>
> You can see the full build log here:
>
>
> https://buildd.debian.org/status/fetch.php?pkg=guile-2.0&arch=armhf&ver=2.0.3%2B1-3&stamp=1327880570
Given that Guile 2.0.5 is working on armhf, I am closing this one as
done. Thanks.
Andy
--
http://wingolog.org/
bug#12827: [PATCH] Tweak web modules, support relative URIs
Daniel Hartwig writes: > On 24 February 2013 18:45, Mark H Weaver wrote: >> I would argue that Absolute-URIs are more often appropriate in typical >> user code. The reason is that outside of URI-handling libraries, most >> code that deals with URIs simply use them as universal pointers, >> i.e. they implicitly assume that each URI is sufficient by itself to >> identify any resource in universe. > > Right. RFC 3986 makes a convincing argument for the new terminology. > These notes about usage also reflect the sentiment in that document. > > FWIW, I sat mostly on the fence, finally going away from URI-Reference > due to these concerns I expressed in an earlier email: >> The API seems less clean, and it is not immediately clear >> that uri? is not the top of the URI-like type hierarchy. The other >> functions only indicate “uri” in their name. I did not >> wish to introduce parallel “build-uri-reference”, etc. for each of >> these, and did consider adding #:reference? on some to select >> weaker validation. > > and looking at some other Scheme URI modules. > > However, having read over your comments I think that we could > reasonably get away with just introducing the procedures you mention > below and not bother about renaming (or duplicating) the field getters > to ‘uri-reference-path’ etc.. Hmm. The cleanest solution would probably be to duplicate the field getters, and make the 'uri-*' variants (e.g. 'uri-path') raise an error when applied to a relative reference. However, it's probably not that important, so if you think it's better to simply extend 'uri-path' etc to apply to all URI-References, I'm okay with that. >> Here's what I suggest: instead of extending 'string->uri' and >> 'build-uri' to produce relative URIs, rename those extended procedures >> 'string->uri-reference' and 'build-uri-reference'. These are long >> names, but that's okay because users should think twice before using >> them, and that's seldom needed. > > In your proposed solution, ‘uri?’ and ‘uri-reference?’ are the > predicates and they respond according to the RFC rather than internal > Guile details? What do you mean by "rather than internal Guile details"? Here's how I like to think about these types: URI-Reference is at the top of the type hierarchy, and URI (a.k.a. Absolute-URI) and Relative-Ref are subtypes. Furthermore, every URI-Reference is either an Absolute-URI or a Relative-Ref. In other words, if you create a URI-Reference that happens to be absolute, then you'll end up with a URI, in the same sense that if you create a complex number whose imaginary part happens to be exact zero, you'll end up with a real number. > That is: > > (uri? (string->uri-reference "http://example.net/";)) > => #t > (uri-reference? (string->uri-reference "http://example.net/";)) > => #t > (uri? (string->uri-reference "foo")) > => #f Yes. >> Then, we extend 'string->uri' and 'build-uri' in a different way: we >> extend them to handle relative references in their *inputs*, but >> continue to provide absolute *outputs*, by adding an optional keyword >> argument '#:base-uri'. This would make it easy to implement the >> simplest and safest strategy outlined above with a minimum of code >> changes. > > This strategy does reflect the recommendation of RFC 3986 to resolve > the references as they are read. Also an elegant API, as it > encourages immedately resolving uri-references and never creating (or > considering to create) the context-sensitive relative-refs. > >> >> What do you think? >> > > I quite like it, particularly the last part about #:base-uri. > > Ludo, I think this is basically what you were suggesting in the first place? > :-) Excellent! BTW, to be clear, I suggest that 'string->uri' and 'build-uri' should be guaranteed to produce Absolute-URIs. In other words, they should raise an error if not given enough information to produce an Absolute-URI. Does that make sense? Thanks again for your work on this :) Mark
bug#10627: char-ready? is broken for multibyte encodings
Hi Andy, Andy Wingo writes: > On Sat 28 Jan 2012 11:21, Mark H Weaver writes: > >> The R5RS specifies that if 'char-ready?' returns #t, then the next >> 'read-char' operation is guaranteed not to hang. This is not currently >> the case for ports using a multibyte encoding. >> >> 'char-ready?' currently returns #t whenever at least one _byte_ is >> available. This is not correct in general. It should return #t only if >> there is a complete _character_ available. > > This procedure is omitted in the R6RS because it is not a good > interface. Besides its semantic difficulties, can you think of a sane > implementation for multibyte characters? Maybe I'm missing something, but I don't see any semantic problem here, and it seems straightforward to implement. 'char-ready?' should simply read bytes until either a complete character is available, or no more bytes are ready. In either case, all the bytes should then be 'unget' before returning. What's the problem? The only reason I haven't yet fixed this is because it will require some refactoring in ports.c. I guess the most straightforward approach is to generalize 'get_codepoint', 'get_utf8_codepoint', and 'get_iconv_codepoint' to support a non-blocking mode of operation. What do you think? Regards, Mark
bug#10627: char-ready? is broken for multibyte encodings
Hi :) On Sun 24 Feb 2013 21:14, Mark H Weaver writes: > Andy Wingo writes: > >> On Sat 28 Jan 2012 11:21, Mark H Weaver writes: >> >>> The R5RS specifies that if 'char-ready?' returns #t, then the next >>> 'read-char' operation is guaranteed not to hang. This is not currently >>> the case for ports using a multibyte encoding. >>> >>> 'char-ready?' currently returns #t whenever at least one _byte_ is >>> available. This is not correct in general. It should return #t only if >>> there is a complete _character_ available. >> >> This procedure is omitted in the R6RS because it is not a good >> interface. Besides its semantic difficulties, can you think of a sane >> implementation for multibyte characters? > > Maybe I'm missing something, but I don't see any semantic problem here, > and it seems straightforward to implement. 'char-ready?' should simply > read bytes until either a complete character is available, or no more > bytes are ready. In either case, all the bytes should then be 'unget' > before returning. What's the problem? The problem is that char-ready? should not read anything. If you want to peek, use peek-char. Note that if the stream is at EOF, char-ready? should return #t. Andy -- http://wingolog.org/
bug#10627: char-ready? is broken for multibyte encodings
Andy Wingo writes: > On Sun 24 Feb 2013 21:14, Mark H Weaver writes: > >> Maybe I'm missing something, but I don't see any semantic problem here, >> and it seems straightforward to implement. 'char-ready?' should simply >> read bytes until either a complete character is available, or no more >> bytes are ready. In either case, all the bytes should then be 'unget' >> before returning. What's the problem? > > The problem is that char-ready? should not read anything. Okay, but if all bytes read are later *unread*, and the reads never block, then why does it matter? The reads in my proposed implementation are just an internal implementation detail, and it seems to me that the user cannot tell the difference, as long as he does not peek underneath the Scheme port abstraction. If you prefer, perhaps a nicer way to think about it is that 'char-ready?' looks ahead in the putback buffer and/or the read buffer (refilling it in a non-blocking mode if needed), and returns #t iff a complete character is present in the buffer(s), or EOF is reached. However, is seems to me that implementing this in terms of read-byte and unget-byte is simpler, because it avoids duplication of the logic regarding putback buffers and refilling of buffers. Maybe there's some reason why this is a bad idea, but I haven't heard one. I agree that 'char-ready?' is an antiquated interface, but it is nonetheless part of the R5RS (and Guile since approximately forever), and it is the only way to do a non-blocking read in portable R5RS. It seems to me that we ought to try to implement it as well as we can, no? > If you want to peek, use peek-char. Okay, but that's a totally different tool with a different use case. It cannot be used to do non-blocking reads. > Note that if the stream is at EOF, char-ready? should return #t. Agreed. More thoughts? Thanks, Mark
bug#13809: Wishlist: support > 10 args to foreign functions
The current limitation of 10 arguments to foreign functions is proving to be a problem for some libraries, in particular the Allegro game library. Is there a reason why raising this limit to 16 or 20 would be undesirable? What tradeoffs are involved? Thanks, Mark
bug#13768: --without-posix code uses scm_getpid() in libguile-2.0.2
reopen 13768 thanks Andy Wingo writes: > On Wed 20 Feb 2013 00:38, Jan Schukat writes: > >> What happens is, in random.c in random_state_of_last_resort on line 668 >> scm_getpid is used to seed the random generator. So either a >> preprocessor switch or a hand constructed scm like in scm_getpid >> (scm_from_ulong(getpid())) should be used there. > > Fixed, thanks for the report. This has potential security implications. If the same program is run multiple times in the same second, then without something like a PID, there's a significant danger that two runs of the program will use the same random seed. Therefore, I think we ought to try hard to ensure that something like a PID will always be included in this seed. Perhaps 'scm_getpid' should be included even when building --without-posix. At the very least, the documentation (which currently claims that the PID is included in the random-state-of-last-resort) should be adjusted to reflect the new reality. I just took care of that. Thanks, Mark
bug#10627: char-ready? is broken for multibyte encodings
On 25 February 2013 08:06, Mark H Weaver wrote: > Andy Wingo writes: > >> On Sun 24 Feb 2013 21:14, Mark H Weaver writes: >> >>> Maybe I'm missing something, but I don't see any semantic problem here, >>> and it seems straightforward to implement. 'char-ready?' should simply >>> read bytes until either a complete character is available, or no more >>> bytes are ready. In either case, all the bytes should then be 'unget' >>> before returning. What's the problem? >> >> The problem is that char-ready? should not read anything. > > Okay, but if all bytes read are later *unread*, and the reads never > block, then why does it matter? Taking care to still use sf_input_waiting for soft ports? Reading bytes from a soft port could have side effects (i.e. logging action or similar).
bug#10015: Guile-2.0.3: test-ffi fails
Hi Hans. Following up on an old bug report: Hans Aberg wrote: > bad return from expression `(f-sum -1 2000 -3 400)': expected > 3971999; got 3972255 > FAIL: test-ffi This turns out to be due to a bug in LLVM, namely that it improperly assumes that signed integer arguments will be sign-extended by the caller, although this is not specified by the SysV x86-64 PS ABI. The recently-released libffi 3.0.12 works around this problem. If you're interested in the gory details, see: http://sourceware.org/ml/libffi-discuss/2013/msg00012.html http://gcc.gnu.org/ml/gcc/2013-01/msg00447.html http://bugs.gnu.org/13342 We've been told by another MacOS user that the FFI tests now pass when Guile is linked against libffi 3.0.12, so I'm closing this bug. Please let us know if you continue to have problems with the FFI on MacOS. Thanks again for the bug report! Mark
