SSL (HTTPS) with 2.4
Hi all. Some time ago (years) I had a script on Python 2.2 that would retieve a HTTPS web site. I used python22-win32-ssl.zip to handle the SSL aspect and it worked wonderfully. I am revisiting the project and need to update it to Python 2.4.1. python22-win32-ssl.zip isn't compatable (duh) and I can't find a newer version. I have had a search and can't find anything to point me in the right direction. Can someone please help? -- http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
Thanks Martin. That means my code should work.
I am trying to go through a proxy, which works fine for HTTP sites.
However, when I try a HTTPS site, the program doesn't respond for quite
a while, and returns the error:
" File "C:\Python24\lib\urllib2.py", line 996, in do_open
raise URLError(err)
URLError: "
Here is my testing code:
import urllib2
proxy_info = {
'user' : 'me',
'pass' : 'mypassword',
'host' : 'proxy.mycompany.com',
'port' : 8008 } # settings taken from web browser
# build a new opener that uses a proxy requiring authorization
proxy_support =
urllib2.ProxyHandler({"https":"https://%(user)s:%(pass)[EMAIL
PROTECTED](host)s:%(port)d"
% proxy_info})
opener = urllib2.build_opener(proxy_support, urllib2.HTTPHandler)
# install it
urllib2.install_opener(opener)
# use it
f = urllib2.urlopen('https://www.directshares.com.au')
print f.headers
f.close()
Any ideas what is wrong?
Bloke
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to learn OO of python?
The best one I found was Dive Into Python - and it's free http://www.diveintopython.org/ Also, How to Think Like a computer scientist - can't remember the link. Bloke -- http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
I just removed my installation of Python 2.4.1, which was the one on
the python.org web site. I installed the Activepython 2.4.1 and now I
get the following error with the same code above:
File "C:\Python24\lib\urllib2.py", line 1053, in unknown_open
raise URLError('unknown url type: %s' % type)
URLError:
I'm getting a bit frustrated. Do I need to import another library?
Any advise is appreciated.
Bloke.
--
http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
Yes, on looking into it, sockets.ssl is not installed with
activepython, so it doesn't recognise https. So I have removed it, and
reinstalled the v 2.4.1 which I downloaded from www.python.org . This
leaves me with with the problem where the script 'hangs' for a long
time, then returns:
Traceback (most recent call last):
File "C:\Documents and
Settings\rhall\Desktop\software\python\auth_example_5.py", line 18, in
?
f = urllib2.urlopen('https://www.directshares.com.au/')
File "C:\Python24\Lib\urllib2.py", line 130, in urlopen
return _opener.open(url, data)
File "C:\Python24\Lib\urllib2.py", line 358, in open
response = self._open(req, data)
File "C:\Python24\Lib\urllib2.py", line 376, in _open
'_open', req)
File "C:\Python24\Lib\urllib2.py", line 337, in _call_chain
result = func(*args)
File "C:\Python24\Lib\urllib2.py", line 1029, in https_open
return self.do_open(httplib.HTTPSConnection, req)
File "C:\Python24\Lib\urllib2.py", line 996, in do_open
raise URLError(err)
URLError:
I'm stuck...
--
http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
I just tried the https connection through a friends internet connection
which uses a transparent proxy as follows:
import urllib2
f = urllib2.urlopen('https://www.directshares.com.au/')
print f.headers
print f.read()
f.close()
This works fine. So it must be a problem with either the proxyhandler
(in Python) or our proxy server. Given that I can connect to the site
through the proxy with a web browser, it must be something to do with
my the way I have set up the proxy handler in python. I have read
multiple ways of setting up the connection, and this is the only way I
can get it to run with http:
import urllib2
proxy_info = {
'user' : 'me',
'pass' : 'password',
'host' : 'companyproxy.com.au',
'port' : 8008 }
# build a new opener that uses a proxy requiring authorization
proxy_support =
urllib2.ProxyHandler({"http":"http://%(user)s:%(pass)[EMAIL
PROTECTED](host)s:%(port)d"
% proxy_info})
opener = urllib2.build_opener(proxy_support, urllib2.HTTPHandler)
# install it
urllib2.install_opener(opener)
Is there a better way for me to do this?
--
http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
Following my above comment, if my script works with http, then what is the problem with https, even when I change the ProxyHandler to specify https? -- http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
Thanks Martin. The problem seems to lie with our company proxy (which requires authentication). I have tried retrieving the page on another network with a transparent proxy, and it all works fine. Unfortnately, any https page I try to retrieve on the company network fails in this way with after a long period of inactivity. However, I can retrieve the page using a standard browser through the same company network. I think there must be something weird going on with our proxy server. -- http://mail.python.org/mailman/listinfo/python-list
urllib2 EOF occurred in violation of protocol with proxy
I want to connect to a https server through an authenticating proxy
like this:
import urllib2
proxy_info = {
'user' : 'me',
'pass' : 'password',
'host' : 'mycompany.com.au',
'port' : 8008 }
# build a new opener that uses a proxy requiring authorization
proxy_support = urllib2.ProxyHandler({"https":"https://%(user)s:%
(pass)[EMAIL PROTECTED](host)s:%(port)d" % proxy_info})
opener = urllib2.build_opener(proxy_support, urllib2.HTTPHandler)
# install it
urllib2.install_opener(opener)
# use it
f = urllib2.urlopen('https://www.securewebsite.com.au')
print f.headers
print f.read()
f.close()
If I modify the 'https' to 'http' it works fine. But in https mode it
returns:
URLError:
The site works fine if I do not go through the proxy, so it appears the
proxy is not returning an EOF. I been searching for an answer, and
found many other sites where people have had the same problem. Some
have suggested m2crypto.
Is there a work around using the standard 2.4.1 installation?
--
http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
Andrew, It seems I'm not the only one going nuts here. I have just spent the last 4 hrs stepping through the code in the debugger. It seems to get stuck somewhere in the socket module (when it calls ssl) but haven't as yet figured out exactly where. I am _very_ interested to find that you have the same prob with a non-authenticating proxy. I had considered I was doing something wrong with the authentication, but from what you say, and from what I have deduced from the code, it is not the authentication that is at fault. Like you, a standard browser works fine, so I'm inclined to think there is something buggy with the way the sockets module talks to the proxy. There has been some suggestion that it may me a 'Microsoftish' proxy which is at fault, but I believe it is a Squid proxy our company uses. There is an interesting note here ( http://www.squid-cache.org/Doc/FAQ/FAQ-11.html setcion 11.34 ) regarding malformed https requests sent through Squid with buggy clients. It may be worth looking into. Anyway, if you have any luck, _please_ let me know - I'm getting desparate. -- http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
Ideally, we should aim at a 'fix' that can be included in the distribution. I am going to look at what communication goes on between the proxy server and a working browser by monitoring the traffic. From what i understand, the proxy needs to be told first to set up a secure connection with the web site, and only then do you pass the url to the proxy. Bloke -- http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
Andrew and John, Any chance of you sending me your code to have a look at? Rob -- http://mail.python.org/mailman/listinfo/python-list
Re: SSL (HTTPS) with 2.4
OK.
I try pyopenssl and can get a secure socket to the server, but am
unsure how to use this socket with urllib2 or even httplib.
Here's the code I'm using:
import sys, socket, string, base64, httplib
from OpenSSL import SSL
# Connects to the server, through the proxy
def run(server, proxy):
user='me';passwd='pass'
#setup basic authentication
if user and passwd:
user_pass=base64.encodestring(user+':'+passwd)
proxy_authorization='Proxy-authorization: Basic
'+user_pass+'\r\n'
else:
proxy_authorization=''
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect(proxy)
print 'Socket established'
except socket.error, e:
print "Unable to connect to %s:%s %s" % (proxy[0], proxy[1],
str(e))
sys.exit(-1)
# Use the CONNECT method to get a connection to the actual server
connectMessage = "CONNECT %s:%s HTTP/1.0\r\n" % (server[0],
server[1]) + \
proxy_authorization #+ 'Proxy-Connection:
Keep-Alive\r\n'
print connectMessage
s.send(connectMessage)
print '\nConnect sent...'
print "Proxy response: %s" % string.strip(s.recv(1024))
ctx = SSL.Context(SSL.SSLv2_METHOD)
conn = SSL.Connection(ctx, s)
# Go to client mode
conn.set_connect_state()
# start using HTTP
conn.send("HEAD / HTTP/1.0\n\n")
print "Server response:"
print "-" * 40
while 1:
try:
buff = conn.recv(4096)
except SSL.ZeroReturnError:
# we're done
break
print buff,
#initalize httplib and replace with your socket
sock = httplib.FakeSocket(s, conn)
print 'Fake socket installed'
h=httplib.HTTPSConnection(server[0],server[1])
h.sock=sock
print 'Sock installed'
h.request('GET','/')
print 'Request sent.'
r=h.getresponse()
print r.read()
if __name__ == '__main__':
server = ('www.anz.com', 443)
proxy = ('proxy.company.com, 8008)
run(server, proxy)
I get the following response at line
59 r=h.getresponse()
Socket established
CONNECT www.anz.com:443 HTTP/1.0
Proxy-authorization: Basic cmhhbGw6YWxlbW0y
Connect sent...
Proxy response: HTTP/1.0 200 Connection established
conn established
conn connect state set
Server response:
HTTP/1.1 200 OK
Server: Microsoft-IIS/4.0
Date: Thu, 26 May 2005 09:33:26 GMT
Content-Type: text /html
Set-Cookie: ASPSESSIONIDCRADCCBB=JPGLOCLDMMFNKJKCMIBADHOH; path=/
Cache-control: private
Fake socket installed
Sock installed
Request sent.
Traceback (most recent call last):
File "C:\Documents and
Settings\rhall\Desktop\software\python\tunnel\proxy-openssl.py", line
65, in ?
run(server, proxy)
File "C:\Documents and
Settings\rhall\Desktop\software\python\tunnel\proxy-openssl.py", line
59, in run
r=h.getresponse()
File "C:\Python24\Lib\httplib.py", line 862, in getresponse
response.begin()
File "C:\Python24\Lib\httplib.py", line 333, in begin
version, status, reason = self._read_status()
File "C:\Python24\Lib\httplib.py", line 291, in _read_status
line = self.fp.readline()
File "C:\Python24\Lib\httplib.py", line 981, in readline
s = self._read()
File "C:\Python24\Lib\httplib.py", line 937, in _read
buf = self._ssl.read(self._bufsize)
ZeroReturnError
I tried enabling 'Proxy-Connection: Keep-Alive' but then it hangs for
ages at:
conn.send("HEAD / HTTP/1.0\n\n")
and eventually returns a 'handshaking' error.
Any pointers anyone?
Rob
--
http://mail.python.org/mailman/listinfo/python-list
Re: dictionaries and threads
I've been trying to find a Python tutorial on threading - does anyone have a reference? Rob -- http://mail.python.org/mailman/listinfo/python-list
Re: dictionaries and threads
thanks. Looks good. -- http://mail.python.org/mailman/listinfo/python-list
