RE: Mechanize: first attempt at scraping (should be something trivial)

2013-03-13 Thread G M
Definitely appears to be network related as I'm getting this when using warnings/fatalsToBrowser: Error GETing http://www.easyjet.com/en/: Can't connect to www.easyjet.com:80 (connect: Connection refused) > Date: Wed, 13 Mar 2013 14:19:00 -0700 > Subject: Re: Mechanize:

Re: Mechanize: first attempt at scraping (should be something trivial)

2013-03-13 Thread Charles DeRykus
On Wed, Mar 13, 2013 at 1:08 PM, G M wrote: > Hi, > > Yeah I tried putting a die line in after doing a bit of googling, I've got a > print "mech ran" line where you've got "die", doesn't print anything out > though :( > > > Hm, the problem is that Mech by default throws fatal errors so if it coul

RE: Mechanize: first attempt at scraping (should be something trivial)

2013-03-13 Thread G M
Hi, Yeah I tried putting a die line in after doing a bit of googling, I've got a print "mech ran" line where you've got "die", doesn't print anything out though :( Cheers, G > Date: Wed, 13 Mar 2013 13:04:39 -0700 > Subject: Re: Mechanize: first

Re: Mechanize: first attempt at scraping (should be something trivial)

2013-03-13 Thread Charles DeRykus
On Wed, Mar 13, 2013 at 12:09 PM, G M wrote: > > Hi all, > > I'm making an attempt at my first screen scraping script. > > For some reason the script doesn't continue after the invocation of the get > method on the last line: > > use strict; > use WWW::Mechanize; > use HTML::TokeParser; > use Dat

Re: Mechanize: first attempt at scraping (should be something trivial)

2013-03-13 Thread Lawrence Statton
On 03/13/2013 01:46 PM, Andy Bach wrote: Wide character in print at /usr/local/bin/mech_test.pl line 18. that's the print page. By the way -- you can eliminate the wide-char warniung by telling perl that your terminal can eat UTF-8 encoded unicode binmode STDOUT, ":utf8"; -- To unsubscri

Re: Mechanize: first attempt at scraping (should be something trivial)

2013-03-13 Thread Andy Bach
On Wed, Mar 13, 2013 at 2:09 PM, G M wrote: > > I'm making an attempt at my first screen scraping script. Works here: !/usr/bin/perl use strict; use WWW::Mechanize; use HTML::TokeParser; use Data::Dumper; print "Content-type: text/html\n\n"; print "setting up mech"; my $agent = WWW::Mechanize->

Re: Mechanize: first attempt at scraping (should be something trivial)

2013-03-13 Thread Jim Gibson
On Mar 13, 2013, at 12:09 PM, G M wrote: > > Hi all, > > I'm making an attempt at my first screen scraping script. > > For some reason the script doesn't continue after the invocation of the get > method on the last line: > > use strict; > use WWW::Mechanize; > use HTML::TokeParser; > use Da

Re: Mechanize Redirect

2008-08-11 Thread Merdinus
On Aug 10, 8:27 am, [EMAIL PROTECTED] (Rob Dixon) wrote: > Merdinus wrote: > > > The webpage is at:  http://patft.uspto.gov/netahtml/PTO/srchnum.htm > > The submit form is submitting all the hidden variables in the webpage > > as well as a 7 digit number for a patent (ex. 7123456).  To gather > > i

Re: Mechanize Redirect

2008-08-10 Thread Rob Dixon
Merdinus wrote: > > The webpage is at: http://patft.uspto.gov/netahtml/PTO/srchnum.htm > The submit form is submitting all the hidden variables in the webpage > as well as a 7 digit number for a patent (ex. 7123456). To gather > info, after the call to "submit_form" I've edited the code to I've

Re: Mechanize Redirect

2008-08-10 Thread Peter Scott
On Sat, 09 Aug 2008 15:47:12 -0700, Merdinus wrote: > The webpage is at: http://patft.uspto.gov/netahtml/PTO/srchnum.htm > The submit form is submitting all the hidden variables in the webpage > as well as a 7 digit number for a patent (ex. 7123456). To gather > info, after the call to "submit_fo

Re: Mechanize Redirect

2008-08-09 Thread Merdinus
On Aug 9, 9:30 am, [EMAIL PROTECTED] (Rob Dixon) wrote: > Merdinus wrote: > > > Have seen this issue addressed on this board in the past, but can't > > figure out exactly what I need to do. > > > I'm using WWW::Mechanize and a webpage seems to be redirecting me but > > Mechanize doesn't seem to fol

Re: Mechanize Redirect

2008-08-09 Thread Rob Dixon
Merdinus wrote: > > Have seen this issue addressed on this board in the past, but can't > figure out exactly what I need to do. > > I'm using WWW::Mechanize and a webpage seems to be redirecting me but > Mechanize doesn't seem to follow it. I've heard to "Add the header > 'Accept: text/html'" but

Re: Mechanize Redirect

2008-08-09 Thread Jeff Pang
Merdinus 写道: > I'm using WWW::Mechanize and a webpage seems to be redirecting me but > Mechanize doesn't seem to follow it. I've heard to "Add the header > 'Accept: text/html'" but alas I don't know how to add the header, or > what that means. > Hi, The $mech->request() method will process red

Re: Mechanize or LWP::RobotUA - which one does it

2008-06-09 Thread Rob Dixon
jobst müller wrote: > > first of all: i am new to the list., > > > i work in the field-research. To begin with: well i have the data in a bunch > of plain text files on the local disk. Well i need to collect some of the > data out of a site - here is an example. > http://www.bamaclubgp.org/

Re: Mechanize::How do I get the results back after Submitting a Form.

2007-12-08 Thread yitzle
$mechObject->content ought to work. For some reason, though, I've been using $mechObject->response()->decoded_content() For the difference, see the mechanize documentation on CPAN -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] http://learn.perl.org

Re: Mechanize + Proxy

2007-10-07 Thread Tom Phoenix
On 10/6/07, yitzle <[EMAIL PROTECTED]> wrote: > How do I direct Mechanize/LWP to visit/click a URL/link via a proxy? Have you seen the FAQ? http://search.cpan.org/dist/WWW-Mechanize/lib/WWW/Mechanize/FAQ.pod#How_do_I_use_WWW::Mechanize_through_a_proxy_server? Hope this helps! --Tom Phoeni

RE: :Mechanize through a proxy?

2004-03-11 Thread Rajesh Dorairajan
Hi James, There should be no problem going by your code and assuming you've the environment variables setup as indicated. What are the contents of $a->content()? Are you getting a error code such as 501, 404? Also, you might want to verify the version of WWW::Mechanize that is installed in your sy

RE: Mechanize 500 connect failed

2003-12-02 Thread Paul Kraus
This is really annoying I get the same issue when I run the script from a Linux server as I do an a windows xp workstation. > -Original Message- > From: Paul Kraus [mailto:[EMAIL PROTECTED] > Sent: Tuesday, December 02, 2003 8:31 AM > To: [EMAIL PROTECTED] > Subject: Mechanize 500 conne

RE: Mechanize

2003-11-14 Thread Wiggins d Anconia
> Ok here is a sample of the data. Its very confidential so I am trying to > be selective on what I send. I am just including the header information > since its not going to be dangerous to send :) > > Basically a table exists on this page that has merchant transaction > info like this... > >

RE: Mechanize

2003-11-14 Thread Paul Kraus
I was way over thinking this. A simply split command to break the scalar content up by \n solved my problems. Paul -Original Message- From: Kevin Old [mailto:[EMAIL PROTECTED] Sent: Friday, November 14, 2003 9:47 AM To: [EMAIL PROTECTED] Cc: 'Beginners Perl' Subject: RE: Mech

RE: Mechanize

2003-11-14 Thread Paul Kraus
Friday, November 14, 2003 9:47 AM To: [EMAIL PROTECTED] Cc: 'Beginners Perl' Subject: RE: Mechanize On Fri, 2003-11-14 at 09:36, Paul Kraus wrote: > Yes I have read the POD in fact I have them printed out in front of > me. Unless I am missing something and I doubt I am since I h

RE: Mechanize

2003-11-14 Thread Kevin Old
On Fri, 2003-11-14 at 09:36, Paul Kraus wrote: > Yes I have read the POD in fact I have them printed out in front of me. > Unless I am missing something and I doubt I am since I have read it like > 4 times. All you can really do using mechanize is parse the content for > links and forms. > > I nee

RE: Mechanize

2003-11-14 Thread Paul Kraus
way I would expect it to behave. The regex I would use is /class="statement" align="center"/ Paul Kraus -Original Message- From: Kevin Old [mailto:[EMAIL PROTECTED] Sent: Friday, November 14, 2003 9:22 AM To: [EMAIL PROTECTED] Cc: 'Beginners Perl' Subject: R

RE: Mechanize

2003-11-14 Thread Kevin Old
On Fri, 2003-11-14 at 09:02, Paul Kraus wrote: > Lol ... Now that is ironic... Should have read ...staring at the code to > long and was missing some obvious typo. :) Paul, Have you looked at the POD docs that came with WWW::Mechanize? It clearly shows how to parse the content http://search.cpa

RE: Mechanize

2003-11-14 Thread Paul Kraus
Lol ... Now that is ironic... Should have read ...staring at the code to long and was missing some obvious typo. :) Paul -Original Message- From: Paul Kraus [mailto:[EMAIL PROTECTED] Sent: Friday, November 14, 2003 8:58 AM To: 'Kevin Old' Cc: 'Beginners Perl' Su

RE: Mechanize

2003-11-14 Thread Paul Kraus
PROTECTED] Sent: Thursday, November 13, 2003 4:51 PM To: [EMAIL PROTECTED] Cc: 'Beginners Perl' Subject: Re: Mechanize On Thu, 2003-11-13 at 16:41, Paul Kraus wrote: > Is it possible to connect to a 128bit encrypted web site with > www::mechinize? > > My boss wants me to write a s

Re: Mechanize

2003-11-13 Thread Kevin Old
On Thu, 2003-11-13 at 16:41, Paul Kraus wrote: > Is it possible to connect to a 128bit encrypted web site with > www::mechinize? > > My boss wants me to write a script that will get a daily report for some > of our financial web sites and merchant accounts. > > This is kind of high priority so th

Re: Mechanize

2003-11-13 Thread Wiggins d Anconia
> Is it possible to connect to a 128bit encrypted web site with > www::mechinize? > > My boss wants me to write a script that will get a daily report for some > of our financial web sites and merchant accounts. > > This is kind of high priority so the sooner you can help the better. > > This i

RE: :Mechanize ->links() problem

2003-09-11 Thread Dan Muey
> > Question: > > I'm trying to use the links(); function which in the man page > > says "returns a list of the links found in the last fetched > > page. " However when I try > > > > @LINKS = $agent->links(); > > foreach(@LINKS){ > > print "$_\n"; > > } > > it returns

RE: :Mechanize ->links() problem

2003-09-11 Thread Wiggins d'Anconia
On Thu, 11 Sep 2003 17:07:14 -0500, "Dan Muey" <[EMAIL PROTECTED]> wrote: > > `perldoc perlref` for a tutorial > That's the reference. You can also do: perldoc perlreftut For a "tutorial". Though I am not sure how recent the Perl needs

RE: :Mechanize ->links() problem

2003-09-11 Thread Dan Muey
> Question: > I'm trying to use the links(); function which in the man page > says "returns a list of the links found in the last fetched > page. " However when I try > > @LINKS = $agent->links(); > foreach(@LINKS){ > print "$_\n"; > } > it returns > > WWW::Mechaniz

RE: :Mechanize ->links() problem

2003-09-11 Thread Hanson, Rob
Each link returned is a WWW::Mechanize::Link object. You need to use the methods supplied to get the info. See: http://search.cpan.org/~petdance/WWW-Mechanize-0.59/lib/WWW/Mechanize/Link.p m Use it like this... @LINKS = $agent->links(); foreach (@LINKS) { print $_->url(), "\n"; print $_->te

RE: :Mechanize and Cookies

2003-09-02 Thread Hanson, Rob
I run into this a lot. There is JavaScript in the page, and you need to emulate that in your script. Look at the source HTML for the page. It takes the password and look like it Base 64 encodes the password, then sets a hidden form field named "encoded_pw" to the value. It then clears the passw