Re: A Question On Libcurl Performance

2013-09-11 Thread David Strauss
On Sat, Aug 31, 2013 at 11:57 AM, Thomas Dineen wrote: > For both Solaris 10 and Fedora 14 Fedora 14 hasn't been supported since 2011, and many of its libraries are very old now. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827

Re: Clarifications on using libcurl

2013-09-13 Thread David Strauss
-parse the XML is also a bundled libcurl example [2]. [1] https://github.com/pantheon-systems/fusedav [2] http://curl.haxx.se/libcurl/c/xmlstream.html -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List

IP address connection fail-over is broken for non-blocking sockets

2013-10-09 Thread David Strauss
n with non-blocking sockets? If not, could there be a flag to force blocking behavior to allow fail-over to occur? -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/lis

Re: libgnurl

2013-10-24 Thread David Strauss
>From the project web page: > In practice, only the OpenSSL and GnuTLS variants seem to see widespread > deployment. Except for *every installation of Fedora, RHEL, CentOS, and Scientific Linux*. --- List admin: http://cool.haxx.se/l

DNS-based cluster awareness for connection pools and pipelines

2013-04-10 Thread David Strauss
I see some exciting pipeline-management features landing in new cURL releases that balance connections to the same hostname. Is there any interest in extending such support to balancing/fail-over between multiple A/ records returned for a domain? Since cURL seems to prefer its own DNS client, i

curl_share and persistent connections

2013-04-10 Thread David Strauss
The share interface documentation [1] specifies that DNS lookups and cookie data get shared, but is there an exhaustive list of what gets shared? Specifically, do persistent connections get shared? [1] http://curl.haxx.se/libcurl/c/libcurl-share.html -- David Strauss | da...@davidstrauss.net

Re: curl_share and persistent connections

2013-04-11 Thread David Strauss
I've sent in a patch to the docs. On Wed, Apr 10, 2013 at 10:31 PM, Nick Zitzmann wrote: > > On Apr 10, 2013, at 6:32 PM, David Strauss wrote: > >> The share interface documentation [1] specifies that DNS lookups and >> cookie data get shared, but is there an ex

Re: certificate verification against system cert (?) when custom CAINFO is set

2013-04-11 Thread David Strauss
What is the output of curl -V? The SSL/TLS library cURL is linked to has a major impact on how it performs system-level validation. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/eti

Re: certificate verification against system cert (?) when custom CAINFO is set

2013-04-11 Thread David Strauss
ace to remove access to any system-level trusted certificates. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl

Re: certificate verification against system cert (?) when custom CAINFO is set

2013-04-11 Thread David Strauss
On Thu, Apr 11, 2013 at 1:39 PM, Daniel Stenberg wrote: > Apple has added some magic for certificate verification in their OpenSSL > version. Apple OS X has a certificate management system that might even be accessible within a chroot. -

Re: Only retrieve the headers of a GET reply and return?

2013-04-11 Thread David Strauss
simply returns the number of bytes sent in, and (3) CURLOPT_PROGRESSFUNCTION (with CURLOPT_NOPROGRESS set to zero) to cancel after the body starts. Part #3 is optional. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [

Re: Only retrieve the headers of a GET reply and return?

2013-04-11 Thread David Strauss
On Thu, Apr 11, 2013 at 1:40 PM, David Strauss wrote: > (3) CURLOPT_PROGRESSFUNCTION (with > CURLOPT_NOPROGRESS set to zero) to cancel after the body starts. Here's a good write-up on how to do that: http://curl.haxx.se/mail/lib-2009-04/0296.html -- David Strauss | da...@david

Re: Only retrieve the headers of a GET reply and return?

2013-04-11 Thread David Strauss
Oh, actually it looks like you can make the transfer "fail" right from the CURLOPT_WRITEFUNCTION, which means you could just have it return zero. You would have to expect libcurl to consider the request failed, though. On Thu, Apr 11, 2013 at 1:46 PM, David Strauss wrote: > On Thu,

Re: DNS-based cluster awareness for connection pools and pipelines

2013-04-12 Thread David Strauss
y using weights. If this were implemented, we would also use it for our PHP and Python API clients, which also connect through load balancers but don't run into as many saturation issues. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --

Re: URL parsing

2013-04-13 Thread David Strauss
On Sat, Apr 13, 2013 at 3:12 AM, Steve Holme wrote: > Whilst I have 20 odd years' experience as a C/C++ developer would someone > be so kind to check the four uses of sscanf() in url.c between lines 4381 > and 4402 to see if this is the best / most optimal way of extracting the > user, password an

Re: Inject a PEM certificate using CURLOPT_SSL_CTX_FUNCTION

2013-04-13 Thread David Strauss
m sure it's also possible programmatically using the OpenSSL API. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: htt

Re: URL parsing

2013-04-13 Thread David Strauss
lly easier to prove correctness of a full fledged lexer/parser or > separate library? I can't see how that can be... A quality lexer/parser guarantees that a specified grammar lacks ambiguity and that crazy/malicious input patterns get handled safely. -- David Strauss | da...@davidstrau

Adding PROPFIND support

2013-04-13 Thread David Strauss
s stream parser: https://github.com/pantheon-systems/fusedav/blob/curl/src/props.c Is there interest in ls-style output for WebDAV, provided the path ends in a slash and an option gets set? -- David Strauss | da...@davidstrauss.net | +1 512 5

Re: Adding PROPFIND support

2013-04-14 Thread David Strauss
like the header write callback, which provides the called function with a more coherent unit of data rather than a buffer of incoming bytes. >From a layering perspective, though, this could all live in a new library that provides libcurl-compatible write callbacks for directory listings that abstract

Re: Adding PROPFIND support

2013-04-15 Thread David Strauss
onsistent listing support to extend to IMAP, DICT, and other cURL protocols, too. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: [PATCH] SFTP file listing suggestion

2013-04-20 Thread David Strauss
libssh2_strerror(err)); > state(conn, SSH_SFTP_CLOSE); > sshc->nextstate = SSH_NO_STATE; > -sshc->actualcode = CURLE_QUOTE_ERROR; > +sshc->actualcode = sftp_libssh2_error_to_CURLE(err); > break; >} >state(conn, SSH_SFTP_NEXT_QUOTE); > > They allow to get better error codes

Re: sharedhandle Curl_resolv_unlock() error!

2013-04-26 Thread David Strauss
o become >>> sftp_libssh2_error_to_CURLE() >>> is not really related to the new callback and I would ask you to submit >>> that >>> as a separate patch (which we could merge at once) >> >> I'm not entirely sure about this one. This would make it impossible to >> tell >> when an error was due to a quote command or when it was due to a >> subse

Re: sharedhandle Curl_resolv_unlock() error!

2013-04-26 Thread David Strauss
Along the mailing list etiquette lines, sorry for my top-post. That's not cool, either. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Verification of Content-Length

2013-04-29 Thread David Strauss
good home in the documentation for what the split in responsibilities is between libcurl and a user of the library? Is there interest in adding optional Content-MD5 support? -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [m

Effect of aborting on slow transfers for slow responses

2013-04-29 Thread David Strauss
Does the counter for CURLOPT_LOW_SPEED_TIME start as soon as the connection establishes, or does it wait until the response starts coming back? I'm curious about the case of a server that takes, say, 60 seconds to prepare the response but sends it back in one burst. -- David Strauss

Re: Verification of Content-Length

2013-04-29 Thread David Strauss
implement this for our own server/client communication and our communication with S3, so the set of implementations needing to interoperate is pretty limited. The goal is to checksum, not to avoid attacks. MD5 is quite adequate for that. I would like to use a standard rather than rolling our own m

Lockup in SSL-based communication

2013-05-01 Thread David Strauss
er.c:1029 #15 0x0033fe82937d in Transfer (conn=0x7fb8c386b600) at transfer.c:1396 #16 Curl_do_perform (data=0x7fb8c387c000) at transfer.c:2108 #17 0x0033fe82982b in Curl_perform (data=data@entry=0x7fb8c387c000) at transfer.c:2232 #18 0x0033fe829d0c in curl_easy_perform (curl=curl@entry=0x7fb8

Re: Lockup in SSL-based communication

2013-05-01 Thread David Strauss
On Wed, May 1, 2013 at 12:28 PM, David Strauss wrote: > What timeouts should we be configuring to give up faster when > it's in this state? Based on empirical data, it looks like CURLOPT_TIMEOUT is taking effect, but I'm curious is there's a more precise way to time out he

Re: Proposed changes to SSL comparison documentation

2013-05-04 Thread David Strauss
S SRP? > * You're missing a full stop at the end of the QSOSSL details line - > "OS/400" should be "OS/400." for consistency ;-) > > I hope my feedback helps > > Steve > --- > List admin: http://cool.haxx.se/list/listinfo/curl-library > Etiquette: http

SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
MEOUT, 60 * 3); Is this an NSS bug, or is it an issue with how libcurl uses NSS? I'm on Fedora 17 with libcurl 7.24.0 and NSS 3.14.3. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
On Tue, May 7, 2013 at 1:46 PM, David Strauss wrote: > NSS seems stuck in poll loop, which has been going on for hours Actually, I'm not sure it's NSS stuck there. The loop could be higher up. I just see an unending series of polls from strace. -- David Strauss | da...@dav

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
This -1 timeout is also in the current master: https://github.com/bagder/curl/blob/master/lib/nss.c#L1518 On Tue, May 7, 2013 at 2:11 PM, David Strauss wrote: > It looks like PR_Recv(conn->ssl[num].handle, buf, (int)buffersize, 0, > -1) in nss_recv() (nss.c) may be the problem. That

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
It looks like PR_Recv(conn->ssl[num].handle, buf, (int)buffersize, 0, -1) in nss_recv() (nss.c) may be the problem. That sets the timeout for NSS to 4294967295. On Tue, May 7, 2013 at 1:57 PM, David Strauss wrote: > On Tue, May 7, 2013 at 1:46 PM, David Strauss wrote: >> NSS seems s

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
://developer.mozilla.org/en-US/docs/PRIntervalTime On Tue, May 7, 2013 at 2:14 PM, David Strauss wrote: > This -1 timeout is also in the current master: > https://github.com/bagder/curl/blob/master/lib/nss.c#L1518 > > On Tue, May 7, 2013 at 2:11 PM, David Strauss wrote: >> It looks like PR_Recv(con

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
late the remaining timeout allowable for the request (considering how DNS, etc, have already contributed to request time) and sending that in. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: h

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
On Tue, May 7, 2013 at 2:44 PM, Daniel Stenberg wrote: > That's already done before the function is called in the first place. The > the GnuTLS and OpenSSL versions of that function for example are completely > non-blocking. Well, then that sounds perfect! -- David S

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
a backport of the fix once it's in: https://bugzilla.redhat.com/show_bug.cgi?id=960765 -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-librar

Re: [bagder/curl] 8ec2cb5544 WIN32 MemoryTracking

2013-05-07 Thread David Strauss
On Tue, May 7, 2013 at 3:10 PM, Mel Smith wrote: > I don't know *how* to revert to an earlier commit :(( git revert 8ec2cb5544 That will do a sort of reverse cherry-pick of that single change. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827

Re: SSL with NSS not properly timing out

2013-05-07 Thread David Strauss
ay 7, 2013 at 2:59 PM, David Strauss wrote: > On Tue, May 7, 2013 at 2:49 PM, Daniel Stenberg wrote: >> Assuming it actually makes any difference for your case at least! ;-) > > If it means that it respects the timeouts we give, it's absolutely a > fix for the problem we see

Re: SSL with NSS not properly timing out

2013-05-08 Thread David Strauss
error immediately if the actual socket is non-blocking, polling it returns EWOULDBLOCK, and the NSS non-blocking property is false. [1] https://bugzilla.redhat.com/show_bug.cgi?id=960765#c2 -- David Strauss | da...@davidstrauss.net | +1

Re: SSL with NSS not properly timing out

2013-05-08 Thread David Strauss
On Wed, May 8, 2013 at 11:29 AM, David Strauss wrote: > which I think it is now I'm referring to very recent releases here, not "now" as in current Fedora packages. -- David Strauss | da...@davidstrauss.net | +1 51

Re: SSL with NSS not properly timing out

2013-05-08 Thread David Strauss
On Wed, May 8, 2013 at 5:01 AM, Kamil Dudka wrote: > Only the handshake still blocks with NSS. Is there a way to fix this, too? Does it at least time out based on the configured libcurl connection or request timeout? -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mob

Re: SSL with NSS not properly timing out

2013-05-08 Thread David Strauss
On Wed, May 8, 2013 at 11:37 AM, David Strauss wrote: > Does it at least time out based on the configured libcurl connection or > request timeout? Read some source and found the answer: yes. timeout = PR_MillisecondsToInterval((PRUint32) time_left); /* Force the handshake now *

Re:

2013-05-10 Thread David Strauss
made some > trivial mistake, if this is the case please let me know, I would greatly > appreciate it. > > source: http://pastie.org/pastes/7827550/text > valgrind output: http://pastie.org/pastes/7827548/text > curl version: 7.30.0 Your trace seems to be missing a lot

Re: IMAP and POP3 with libcurl 7.30

2013-05-12 Thread David Strauss
y here, I can say that the command-line curl uses libcurl. If something is possible with the curl CLI, it's possible with libcurl. There are a couple of cases where the curl CLI provides additional logic, like HTTP retries, on top of libcurl. But, it's never redundant with what libcurl

Re: how curl get access_token in OAuth?

2013-05-14 Thread David Strauss
u want something like curl_easy_getinfo() with CURLINFO_EFFECTIVE_URL [2] [1] http://liboauth.sourceforge.net/ [2] http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [

Re: curl-library Digest, Vol 93, Issue 27

2013-05-14 Thread David Strauss
On Tue, May 14, 2013 at 9:46 PM, Aldrich wrote: > OK,I will try it. That doesn't have any context as a reply to a digest. Required reading: http://curl.haxx.se/mail/etiquette.html -- David Strauss | da...@davidstrauss.net | +1 512 577 5827

Re: Read Certificates from curl

2013-05-17 Thread David Strauss
#x27;t require you to fully manage the SSL context, but it is not portable to non-OpenSSL backends for libcurl. [1] http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTSSLCTXFUNCTION -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --

Re: axtls memory leak

2013-05-17 Thread David Strauss
On Thu, May 16, 2013 at 2:10 PM, Hu, Eric wrote: > Regarding the patch itself, it still blocks in the connect phase. This is true for the NSS SSL back-end, too. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mob

Interest in an Expat XML example?

2013-05-17 Thread David Strauss
or any XML. -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: Interest in an Expat XML example?

2013-05-19 Thread David Strauss
I've submitted an XML stream parsing example as a pull request: https://github.com/bagder/curl/pull/66 I dropped the PROPFIND stuff for clarity. It's probably best to keep it to basics for this first example. --- List admin: http://co

Re: Interest in an Expat XML example?

2013-05-19 Thread David Strauss
I wanted to note that this is far more efficient than the existing XML/HTML parsing examples. The existing examples pull the entire response into memory before starting to parse. This new example only allocates buffers to fit the character data between tags. This is pretty important for our use ca

Re: Interest in an Expat XML example?

2013-05-19 Thread David Strauss
On Sun, May 19, 2013 at 2:22 AM, Daniel Stenberg wrote: > ... and I've merged and pushed it already. Thanks! Awesome! I verified that everything is clean using cppcheck, Clang Analyzer, and Valgrind, so it should be solid. -- David Strauss | da...@davidstrauss.net | +1 512

Re: header functions and error response headers

2013-05-27 Thread David Strauss
l/c/curl_easy_getinfo.html#CURLINFORESPONSECODE -- David Strauss | da...@davidstrauss.net | +1 512 577 5827 [mobile] --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re:

2013-05-27 Thread David Strauss
transfer by returning 0 "pre-maturely" (i.e before the server expected it, like when you've said you will upload N bytes and you upload less than N bytes), you may experience that the server "hangs" waiting for the rest of the data that won't come. [1] http:

Re: Patch: Ability to set options for hashing, HTTP Content-MD5 support, Supporting Digest qop=auth-int part 1

2013-05-28 Thread David Strauss
I know Daniel has his reservations, but I would love to see optional Content-MD5 support in libcurl. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html

Re: Regarding Usage of Custom Application Layer in Internet Protocol Suite

2013-05-31 Thread David Strauss
You could also just use an SSL sockets library directly or look at ZeroMQ. libcurl isn't suited to using HTTP without the HTTP. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquet

Re: question

2013-07-07 Thread David Strauss
Yes, you should use your package manager (Yum). I'd like to note, though, that Fedora 16 is not a supported release. Yum mirror availability may be spotty. --- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http

Re: New SSL backend GSKit, certinfo for everyone, etc.

2013-07-12 Thread David Strauss
OS400 version >= 7.1). > - QsoSSL is obsolescent. > > Your comments are welcome. If nobody objects, I'll commit it in a few > days. > > Good week-end to all of you, > Patrick > > --- > List