hiya everyone, I've made this little webcrawler using BeautifulSoup, urllib and urllib2. I've been encountering ASP.NET Forms recently and I can't seem to make a proper post to some of them. My post function has been doing great until this particular website. Here's some explanations first so that you guys understands my problem...
Usually there are 8 hidden inputs spread all over the web page: __EVENTTARGET --> that is the id of the control that is assigned by the ASP.NET engine. It is usually in this particular form "dg_Result$_ctl2$linkbtn1" The javascript function DoPostback usually replaces the '$' for a ':' before doing the real post. Some other ASP.NET page doesn't. __EVENTARGUMENT --> nothing's here usually. __ScrollTop --> its value is often 0 __ScrollLeft --> same as __ScrollTop __ValidationSummary --> usually empty __VIEWSTATEENCRYPTED --> an encrypted string. I don't know what it means nor what it does. __EVENTVALIDATION --> an encrypted string. I don't know what it means nor what it does. __VIEWSTATE --> the encrypted state of the current control. the state of a datagrid for example. I extract all those values using regexes on the web page, build a dictionnary out of all those values and encrypt the whole thing using urllib.urlencode(). Afterwards I extract the form action and post the whole thing with cookies enabled. This procedure works like a charm. The website I've just encounted has only 3 of the 8 hidden inputs. (EVENTTARGET, EVENTARGUMENT & VIEWSTATE ). I tried posting the whole thing using only these 3 values & nada. I then tried posting the whole thing using all empty values for the 5 remaining values...still nothing. Has anyone tried what I'm doing? and if you tried how have you succeeded getting the data back after the post action? thanks for any tips! cP -- http://mail.python.org/mailman/listinfo/python-list
