http-read with cookie

2012-05-06 Thread Sunjoong Lee
Hello,

There is http-get in Guile and will be http-put, then what's the next? I
think the method using cookie and handling https. I had thought that I'd be
able to read contents of a web page needing login when I had read http-put
of Greg Benison,
http://lists.gnu.org/archive/html/guile-user/2012-04/msg00012.html .
How naive am I.

I recognize we need handling https because most login process of web pages
require it but have no idea of it now. During some trial, I made a very
little effort and want to share it. Of course, there shold be the better
way; I'm a newbie yet.

1.1 Exampe of http-get:

(receive (header body)
(http-get (string->uri "http://code.google.com/";))
  (display header)
  (newline))
(receive (header body)
(http-get (string->uri "http://code.google.com/";))
  (display header)
  (newline))

1.2 Output of above example:

#< version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((vary
accept-language cookie referer) (content-type text/html (charset .
"ISO-8859-1")) (etag "fe11bb5c119a4c282ecbc99fc66bc7f1" . #t)
(last-modified . #) (date . #) (expires .
#) (cache-control private (max-age . 3600))
(x-content-type-options . "nosniff") (set-cookie .
"PREF=ID=6d783c184a55e937:TM=1336287789:LM=1336287789:S=p7v3fPvc55hTb15k;
expires=Tue, 06-May-2014 07:03:09 GMT; path=/; domain=.google.com") (server
. "codesite_static_content") (x-xss-protection . "1; mode=block")
(x-frame-options . "SAMEORIGIN") (connection close)) port: #>
#< version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((vary
accept-language cookie referer) (content-type text/html (charset .
"ISO-8859-1")) (etag "fe11bb5c119a4c282ecbc99fc66bc7f1" . #t)
(last-modified . #) (date . #) (expires .
#) (cache-control private (max-age . 3600))
(x-content-type-options . "nosniff") (set-cookie .
"PREF=ID=e7e80585317f6678:TM=1336287790:LM=1336287790:S=PQWgTbaxsZyIqeVy;
expires=Tue, 06-May-2014 07:03:10 GMT; path=/; domain=.google.com") (server
. "codesite_static_content") (x-xss-protection . "1; mode=block")
(x-frame-options . "SAMEORIGIN") (connection close)) port: #>

1.3 Discussion about above output:
1) We need a web page using cookie. I picked http://code.google.com/ ; it
set PREF cookie once unlike http://www.google.com/ twice.
2) Example displays two headers, so the output is two line. In the first
line, http://code.google.com/ replied with set-cookie PREF. You may want to
send cookie header when the next request. If not so, the server would
re-send set-cookie PREF again as seen in second line.
3) Cookie header is essential when making a session. In above example, of
course, actually no need to making a session, so no need cookie, it's just
a test.

2.1 Example of http-read:

(receive (header body)
(http-read "http://code.google.com/";)
  (display header)
  (newline))
(receive (header body)
(http-read "http://code.google.com/";)
  (display header)
  (newline))

2.2 Output of above example:

#< version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((vary
accept-language cookie referer) (content-type text/html (charset .
"ISO-8859-1")) (etag "fe11bb5c119a4c282ecbc99fc66bc7f1" . #t)
(last-modified . #) (date . #) (expires
. #) (cache-control private (max-age . 3600))
(x-content-type-options . "nosniff") (set-cookie .
"PREF=ID=8253d36f6bcd8b99:TM=1336289038:LM=1336289038:S=xDYcthWhrVAC2asa;
expires=Tue, 06-May-2014 07:23:58 GMT; path=/; domain=.google.com") (server
. "codesite_static_content") (x-xss-protection . "1; mode=block")
(x-frame-options . "SAMEORIGIN") (connection close)) port: #>
#< version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((vary
accept-language cookie referer) (content-type text/html (charset .
"ISO-8859-1")) (etag "fe11bb5c119a4c282ecbc99fc66bc7f1" . #t)
(last-modified . #) (date . #) (expires
. #) (cache-control public (max-age . 3600))
(x-content-type-options . "nosniff") (server . "codesite_static_content")
(x-xss-protection . "1; mode=block") (x-frame-options . "SAMEORIGIN")
(connection close)) port: #>

2.3 Discussion about above output:

1) http-read can accept a string URL as it's first argument; of course, it
can accept a uri object like result of string->uri.
2) In the first line, the server replied with set-cookie PREF like 1.2 but
there is no second set-cookie PREF in the second line because http-read had
sent cookie header when second request.

I had attatched a code of http-read.


http-read.scm.gz
Description: GNU Zip compressed data


Re: Working http-get example and still unsolved problem

2012-05-06 Thread Sunjoong Lee
Ian had made a new patch for Chunked Encoding,
http://lists.gnu.org/archive/html/guile-devel/2012-05/msg00040.html ;
thanks Ian. But I suspect it has a bug.

make-chunked-input-port use bytevector-copy! in read!. I saw the crash like
this;
  ERROR: In procedure bytevector-copy!:
  ERROR: Value out of range 0 to 4294967295: -5915

I can't tell the url of the web page made that crash because I had seen it
in my testing login to that page but can tell you that was not written by
english and the charset of that is not UTF-8. I suspect the cause of the
crash be in using bytevector-copy!.

The previous patch made in 06 Nov 2011 does not make a crash. With that, I
could not read the content body of the page but it might be my fault.

2012/5/2 Sunjoong Lee 
>
> 1. http-get of current stable version, 2.0.5,  of Guile does not
> support "Chunked Encoding."
> 2. You should apply patches of Ian Price's,
> http://lists.gnu.org/archive/html/guile-user/2011-11/msg00011.html , if
> this case.
>


Re: Working http-get example and still unsolved problem

2012-05-06 Thread Ian Price
Sunjoong Lee  writes:

> Ian had made a new patch for Chunked 
> Encoding, http://lists.gnu.org/archive/html/guile-devel/2012-05/msg00040.html 
> ; thanks Ian. But I suspect it has a bug.
>
> make-chunked-input-port use bytevector-copy! in read!. I saw the crash like 
> this;
>   ERROR: In procedure bytevector-copy!:
>   ERROR: Value out of range 0 to 4294967295: -5915
>
> I can't tell the url of the web page made that crash because I had seen it in 
> my testing login to that page but can tell you that was not written by 
> english and the charset of
> that is not UTF-8. I suspect the cause of the crash be in using 
> bytevector-copy!.

Odd that I hadn't noticed this issue when I was testing. The problem
occurs in the case where the number to be read is less than the
remaining size of a chunk. I was updating the count with the buffer
size, rather than the difference, which leads to negative values on the
recursive call. There was also a mistake in the same case in updating
the buffer-pointer.

Thanks for pointing this out, and if you have other comments on the
implementation (and the close-port issue), I'll be happy to have them.

-- 
Ian Price

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"