How to share session with IE
Hello! I need to process some webpages of a forum which is powered by discuz!. When I login, there are some options about how long to keep the cookies: forever, month, week, et al. If I choose forever, I don't need to login each time, and When I open the internet explorer I can access any pages directly. Some urls of the pages like: http://www.somesite.com/bbs/viewthread.php?tid=12345&extra=page%3D1 However, now I need to process some pages by a python program. When I use urllib.urlopen(theurl), I can only get a page which told me I need login. I think It's reasonable, becuase I wasn't in a loggined session which as IE did. So how can I do my job? I want to get the right webpage by the url. I have search answers from the groups but didn't get clear answer. Should I use win32com or urllib? Any reply or information is appreciate. Hope I put it clear. Dapu -- http://mail.python.org/mailman/listinfo/python-list
Re: How to share session with IE
It's exactly what I want. I'll try. Thanks! Bernard wrote: > Hello Dapu, > > You can do the same thing as IE on your forum using urllib2 and > cookielib. In short you need to code a small webcrawler. I can give you > my browser module if necessary. > You might not have the time to fiddle with the coding part or my > browser module so you can also use this particularly useful module : > http://wwwsearch.sourceforge.net/mechanize/ > The documentation is pretty clear for an initiated python programmer. > If it's not your case, I'd recommend to read some ebooks on the python > language first to get use to it. > > Bernard > > > > > zdp wrote: > > Hello! > > > > I need to process some webpages of a forum which is powered by discuz!. > > When I login, there are some options about how long to keep the > > cookies: forever, month, week, et al. If I choose forever, I don't > > need to login each time, and When I open the internet explorer I can > > access any pages directly. Some urls of the pages like: > > > > http://www.somesite.com/bbs/viewthread.php?tid=12345&extra=page%3D1 > > > > However, now I need to process some pages by a python program. When I > > use urllib.urlopen(theurl), I can only get a page which told me I need > > login. I think It's reasonable, becuase I wasn't in a loggined session > > which as IE did. > > > > So how can I do my job? I want to get the right webpage by the url. I > > have search answers from the groups but didn't get clear answer. Should > > I use win32com or urllib? Any reply or information is appreciate. Hope > > I put it clear. > > > > Dapu -- http://mail.python.org/mailman/listinfo/python-list
Re: How to share session with IE
I found some similar topics in the newsgroup and get some ideas from
them.
http://groups.google.com/group/comp.lang.python/browse_thread/thread/2fe0be6c386adce4
http://groups.google.com/group/comp.lang.python/browse_thread/thread/a51cec8747f64619
According to all you suggestions, there are at least two ways to get my
result.
1. Use the cookie of IE, so I don't need to code to logon. That means I
must use ClientCookie. I found some example in the docs and the
newsgroup. Below is some code based on the docs of ClientCookie. But
the page I get is still the page told me must login ( I CAN get the
right page in IE).
import ClientCookie, urllib2
url_string="http://www.targetsite.com/bbs/viewthread.php?tid=12345";
#the page I want to get
cj = ClientCookie.MSIECookieJar(delayload=True)
cj.load_from_registry()
print cj #I want to know what I get
opener =
ClientCookie.build_opener(ClientCookie.HTTPCookieProcessor(cj))
ClientCookie.install_opener(opener)
f = ClientCookie.urlopen(url_string)
print f.read() # NOT the right page html
2. Logon myself by python. First, I access the login page and submit
the form of username and password. The form has many fields other than
username and passwd, so the dict "data" has all the fields even if it's
hide. Then, if the login succeed, I can get my page use the opener with
CookieJar.
import urllib2, cookielib
url_string="http://www.targetsite.com/bbs/viewthread.php?tid=12345";
#the page I want to get
url_login="http://www.targetsite.com/bbs/logging.php?action=login";
#the login page
headers = {'User-agent' : 'Mozilla/4.0 (compatible; MSIE 5.5;
Windows NT)'}
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
urllib2.install_opener(opener)
data = {
'formhash': '3bd8bc0a',
"referer" : "index.php",
"loginfield": "username",
'username': 'myname',
'password': 'mypass',
"questionid": 0,
"answer":"",
"cookietime" : "31536",
"loginmode":"",
"styleid":""
}
req=urllib2.Request(url_login, urllib.urlencode(data), headers)
f = opener.open(req)
print req.get_data()
print req.header_items()
print f.info()
print f.read()
## if login succeed, I can get my page
f=opener.open( url_string)
However, both ways didn't work for me. I don't know what's wrong. If
it's because the server page check the header or the submit of the form
is wrong?
I didn't study Mechanize module yet. I want a solution as simple as
possible for distribution reason.
John J. Lee 写道：
> "Bernard" <[EMAIL PROTECTED]> writes:
> > zdp wrote:
> [...]
> > > However, now I need to process some pages by a python program. When I
> > > use urllib.urlopen(theurl), I can only get a page which told me I need
> > > login. I think It's reasonable, becuase I wasn't in a loggined session
> > > which as IE did.
> > >
> > > So how can I do my job? I want to get the right webpage by the url. I
> > > have search answers from the groups but didn't get clear answer. Should
> > > I use win32com or urllib? Any reply or information is appreciate. Hope
> > > I put it clear.
>
> > You can do the same thing as IE on your forum using urllib2 and
> > cookielib. In short you need to code a small webcrawler. I can give you
> > my browser module if necessary.
> > You might not have the time to fiddle with the coding part or my
> > browser module so you can also use this particularly useful module :
> > http://wwwsearch.sourceforge.net/mechanize/
> > The documentation is pretty clear for an initiated python programmer.
> > If it's not your case, I'd recommend to read some ebooks on the python
> > language first to get use to it.
>
> In particular, if you're following the approach Bernard suggests, you
> can either:
>
> 1. Log in every time your program runs, by going through the sequence
>of clicks, pages, etc. that you would use in a browser to log in.
>
> 2. Once only (or once a month, or whatever), log in by hand using IE
>with a "Remember me"-style feature (if the website offers that) --
>where the webapp asks the browser to save the cookie rather than
>just keeping it in memory until you close your browser. Then your
>program can load the cookies from your real browser's cookie store
>using this:
>
> http://wwwsearch.sourceforge.n
Py2exe make wxPython window looks bad
Dear all: I made a window program by wxPython. Split windows, treectrl, listctrl and textctrl are used. When I program in python, the look & feel of the window controls are like the windos XP look & feel, with thin and flat border (My os is window XP). It's natural because, as I know, wxPython use native look and feel. But when I convert the program to EXE file by Py2exe, and run it, the look & feel is bad. It's just like the windows 9x. All controls has a thick and emboss border. However, the scrollbars look good, same as the window XP. So I'm puzzled. What's the problem and how can I get a XP look & feel window program? Should I add some code in my program to set the look and feel, or, set some options in py2exe script to force the generated exe has specified look and feel? The setup.py is very simple as following: -setup.py- import sys, os from distutils.core import setup import py2exe import glob setup( name = "mainframe", windows = ["mainframe.py"] ) -- http://mail.python.org/mailman/listinfo/python-list
Re: Py2exe make wxPython window looks bad
Great, thanks all [EMAIL PROTECTED] 写道： > zdp wrote: > > Dear all: > > > > I made a window program by wxPython. Split windows, treectrl, listctrl > > and textctrl are used. When I program in python, the look & feel of the > > window controls are like the windos XP look & feel, with thin and flat > > border (My os is window XP). It's natural because, as I know, wxPython > > use native look and feel. > [snip] > > http://wiki.wxpython.org/index.cgi/FAQ#head-4cc058aed6216dd200d55a6e4c077ccbe82bd142 > > Regards, > > Ray Smith > http://RaymondSmith.com -- http://mail.python.org/mailman/listinfo/python-list
WebBrowser: How to cast the document object
Hi, all,
My project is based on wxPython, and I need an IE control (i.e.
WebBrowser ActiveX control). Although the wxPython implements a
wrapped version (wx.lib.iewin.IEHtmlWindow), but it doesn't meet all
my demands, because I need to custom many behaviors of the control.
So I thought I should use it through ActiveXWrapper directly.
So I use makepy to make the typelib of "Microsoft Internet Controls"
and "Microsoft HTML Object Library". Now I can get the document object
of the WebBrowser, and I can cast it into any interface I need like
IHtmlDocument2, IHtmlDocument3... So far, so good.
The document object also implements a COM interface
IPersistStreamInit, which has a *Load* method. Calling this method can
load any stream into the browser control. Here is the a example using
this method in visual c++:
IPersistStreamInit* spPSI = NULL;
CStreamOnCString stream(szHTML);
if (m_pHtmlDoc) {
m_hResult = m_pHtmlDoc->QueryInterface(IID_IPersistStreamInit,
(void**)&spPSI);
if( SUCCEEDED(m_hResult) && spPSI ) {
m_hResult = spPSI->Load(static_cast(&stream));
spPSI->Release();
}
}
Now I need to call this method on my document object, my code is just
like
stream = win32com.client.CastTo(doc, "IPersistStreamInit")
stream.Load(somestr)
But I got an error:
ValueError: The interface name 'IPersistStreamInit' does not appear in
the same
library as object ''
I googled and someones says only interfaces inherit from IDispatch can
be used by pythoncom. But IPersistStreamInit interface inherits from
IUnknown. But I just need to use
this interface.
However, I also noted that the class wx.lib.iewin.IEHtmlWindow has a
*LoadStream* method, I thought it's a wrapper of IPersistStreamInit's
Load method. So I trace the source code, but only found the implement
is in the c++ base class.
Could anybody tell me how to do it? Any clues is helpful.
Dapu
--
http://mail.python.org/mailman/listinfo/python-list
Your message to ZDP awaits moderator approval
Your mail to 'ZDP' with the subject hi, ive a new mail address Is being held until the list moderator can review it for approval. The reason it is being held: Post by non-member to a members-only list Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: http://mail.zope.org/mailman/confirm/zdp/15b4d34d8649c773657bbb772cf758376d7c3ca1 -- http://mail.python.org/mailman/listinfo/python-list
Your message to ZDP awaits moderator approval
Your mail to 'ZDP' with the subject hi, ive a new mail address Is being held until the list moderator can review it for approval. The reason it is being held: Post by non-member to a members-only list Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: http://mail.zope.org/mailman/confirm/zdp/b285c9e2b9144e6c7d97399c7d34be95a8c4b44c -- http://mail.python.org/mailman/listinfo/python-list
