Re: urllib unqoute providing string mismatch between string found using os.walk (Python3)

2019-12-22 Thread Ben Hearn
On Saturday, 21 December 2019 21:46:43 UTC, Ben Hearn wrote: > Hello all, > > I am having a bit of trouble with a string mismatch operation in my tool I am > writing. > > I am comparing a database collection or url quoted paths to the paths on the > users drive. > > These 2 paths look identic

Re: urllib unqoute providing string mismatch between string found using os.walk (Python3)

2019-12-21 Thread Richard Damon
On 12/21/19 8:25 PM, MRAB wrote: > On 2019-12-22 00:22, Michael Torrie wrote: >> On 12/21/19 2:46 PM, Ben Hearn wrote: >>> These 2 paths look identical, one from the drive & the other from an >>> xml url: >>> a = '/Users/macbookpro/Music/tracks_new/_NS_2018/J.Staaf - >>> ¡Móchate! _PromoMix_.wav'

Re: urllib unqoute providing string mismatch between string found using os.walk (Python3)

2019-12-21 Thread MRAB
On 2019-12-22 00:22, Michael Torrie wrote: On 12/21/19 2:46 PM, Ben Hearn wrote: These 2 paths look identical, one from the drive & the other from an xml url: a = '/Users/macbookpro/Music/tracks_new/_NS_2018/J.Staaf - ¡Móchate! _PromoMix_.wav'

Re: urllib unqoute providing string mismatch between string found using os.walk (Python3)

2019-12-21 Thread Chris Angelico
On Sun, Dec 22, 2019 at 11:33 AM Michael Torrie wrote: > > On 12/21/19 2:46 PM, Ben Hearn wrote: > > These 2 paths look identical, one from the drive & the other from an xml > > url: > > a = '/Users/macbookpro/Music/tracks_new/_NS_2018/J.Staaf - ¡Móchate! > > _PromoMix_.wav' >

Re: urllib unqoute providing string mismatch between string found using os.walk (Python3)

2019-12-21 Thread Michael Torrie
On 12/21/19 2:46 PM, Ben Hearn wrote: > These 2 paths look identical, one from the drive & the other from an xml url: > a = '/Users/macbookpro/Music/tracks_new/_NS_2018/J.Staaf - ¡Móchate! > _PromoMix_.wav' ^^ > b = '/Users/macbookpro

Re: urllib unqoute providing string mismatch between string found using os.walk (Python3)

2019-12-21 Thread Dan Sommers
On 12/21/19 4:46 PM, Ben Hearn wrote: import difflib print('\n'.join(difflib.ndiff([a], [b]))) - /Users/macbookpro/Music/tracks_new/_NS_2018/J.Staaf - ¡Móchate! _PromoMix_.wav ? ^^ + /Users/ma

Re: urllib unqoute providing string mismatch between string found using os.walk (Python3)

2019-12-21 Thread Pieter van Oostrum
Ben Hearn writes: > Hello all, > > I am having a bit of trouble with a string mismatch operation in my tool I am > writing. > > I am comparing a database collection or url quoted paths to the paths on the > users drive. > > These 2 paths look identical, one from the drive & the other from an xm

urllib unqoute providing string mismatch between string found using os.walk (Python3)

2019-12-21 Thread Ben Hearn
Hello all, I am having a bit of trouble with a string mismatch operation in my tool I am writing. I am comparing a database collection or url quoted paths to the paths on the users drive. These 2 paths look identical, one from the drive & the other from an xml url: a = '/Users/macbookpro/Music

Re: urllib

2017-06-30 Thread breamoreboy
On Friday, June 30, 2017 at 1:30:10 PM UTC+1, Rasputin wrote: > good luck with that, mate ! Please don't change the subject line and also provide some context when you reply, we're not yet mindreaders :) Kindest regards. -- Mark Lawrence. -- https://mail.python.org/mailman/listinfo/python-list

urllib

2017-06-30 Thread Rasputin
good luck with that, mate ! -- https://mail.python.org/mailman/listinfo/python-list

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-21 Thread Jon Ribbens
On 2016-06-21, Steven D'Aprano wrote: > "In our case, if we could fool an internal Python application into fetching > a URL for us, then we could easily access memcached instances. Consider the > URL: ..." > > and then they demonstrate an attack against memcache. Except, the author of > the articl

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-21 Thread Steven D'Aprano
On Sun, 19 Jun 2016 03:28 am, Random832 wrote: > On Sat, Jun 18, 2016, at 12:02, Steven D'Aprano wrote: >> Er, you may have missed that I'm talking about a single user setup. >> Are you suggesting that I can't trust myself not to forge a request >> that goes to a hostile site? >> >> It's all well

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-18 Thread Paul Rubin
Steven D'Aprano writes: >> The issue ... is cross-site request forgery. > Er, you may have missed that I'm talking about a single user setup. Are you > suggesting that I can't trust myself not to forge a request that goes to a > hostile site? I think the idea is you visit some website with malici

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-18 Thread Random832
On Sat, Jun 18, 2016, at 12:02, Steven D'Aprano wrote: > Er, you may have missed that I'm talking about a single user setup. > Are you suggesting that I can't trust myself not to forge a request > that goes to a hostile site? > > It's all well and good to say that the application is vulnerable to >

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-18 Thread alister
On Sun, 19 Jun 2016 02:02:43 +1000, Steven D'Aprano wrote: > On Sat, 18 Jun 2016 01:52 pm, Random832 wrote: > >> On Fri, Jun 17, 2016, at 21:00, Steven D'Aprano wrote: >>> The author doesn't go into details of what sort of attacks against >>> localhost they're talking about. An unauthenticated se

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-18 Thread Steven D'Aprano
On Sat, 18 Jun 2016 01:52 pm, Random832 wrote: > On Fri, Jun 17, 2016, at 21:00, Steven D'Aprano wrote: >> The author doesn't go into details of what sort of attacks against >> localhost they're talking about. An unauthenticated service running on >> localhost implies, to me, a single-user setup,

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-18 Thread Marko Rauhamaa
Steven D'Aprano : > "Even an unauthenticated service listening on localhost is risky these > days." > > but fall short of *explicitly* recommending that they should be > authenticated. Although they do *implicitly* do so, by saying that "it > wouldn't be hard" for such services to include a passwo

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-17 Thread Random832
On Fri, Jun 17, 2016, at 21:00, Steven D'Aprano wrote: > The author doesn't go into details of what sort of attacks against > localhost they're talking about. An unauthenticated service running on > localhost implies, to me, a single-user setup, where presumably the > single-user has admin access t

Re: (repost) Advisory: HTTP Header Injection in Python urllib

2016-06-17 Thread Steven D'Aprano
On Sat, 18 Jun 2016 04:49 am, Paul Rubin wrote: > The blog post below is from a couple days ago: > > http://blog.blindspotsecurity.com/2016/06/advisory-http-header-injection-in.html > The blog post criticizes Redis and Memcached for not using any > authentication (since "safe" internal networks

(repost) Advisory: HTTP Header Injection in Python urllib

2016-06-17 Thread Paul Rubin
The blog post below is from a couple days ago: http://blog.blindspotsecurity.com/2016/06/advisory-http-header-injection-in.html It reports that it's possible to inject fake http headers into requests sent by urllib2(python2) and urllib(python3), by getting the library to retrieve a url conc

Re: error reading api with urllib

2015-12-17 Thread Simian
I will try adding the get. I have not used curl. I also forgot to mention that the code runs against another server, though a slightly different version number. Thanks to you both. Simian -- https://mail.python.org/mailman/listinfo/python-list

Re: error reading api with urllib

2015-12-16 Thread Ian Kelly
On Wed, Dec 16, 2015 at 3:12 PM, John Gordon wrote: > In <9aa21642-765b-4666-8c66-a6dab9928...@googlegroups.com> > simian...@gmail.com writes: > >> Bad Request >> b'' > > > That probably means you aren't using one of the recognized methods > (i.e. GET, POST, etc.) > > It doesn't look like you are

Re: error reading api with urllib

2015-12-16 Thread Ian Kelly
On Tue, Dec 15, 2015 at 7:46 PM, Simian wrote: > I added > > except urllib.error.HTTPError as e: > print('HTTP Errpr') > print('Error code: ', e.code) > > to my try and I recieve... > > 400: ('Bad Request', > 'Bad request syntax or unsupported method'), > > but processing the string

Re: error reading api with urllib

2015-12-16 Thread John Gordon
In <9aa21642-765b-4666-8c66-a6dab9928...@googlegroups.com> simian...@gmail.com writes: > Bad Request > b'' That probably means you aren't using one of the recognized methods (i.e. GET, POST, etc.) It doesn't look like you are specifying one of these methods on your Request object. Try doing t

Re: error reading api with urllib

2015-12-15 Thread Simian
I added except urllib.error.HTTPError as e: print('HTTP Errpr') print('Error code: ', e.code) to my try and I recieve... 400: ('Bad Request', 'Bad request syntax or unsupported method'), but processing the string with a browser works fine. Simi -- https://mail.python.org/mailm

error reading api with urllib

2015-12-15 Thread simian336
Hi, I am pretty new to python. I have a project reading an api with urllib. The problem is I have to sections of code almost exactly the same. The first url works great. They second one fails. If I manually copy and paste the url in the browser ti works great. The error I get back is... Bad

Re: No request in module urllib ?

2015-09-03 Thread Mark Lawrence
On 04/09/2015 03:30, Chris Angelico wrote: On Fri, Sep 4, 2015 at 12:17 PM, Vincent Vande Vyvre wrote: import urllib.request urllib.request.urlopen('http://example.org') Thanks, that works with 3.4.0. No with 3.2.3 Hmm, not sure why it wouldn't. According to the docs [1] it should be avail

Re: No request in module urllib ?

2015-09-03 Thread Vincent Vande Vyvre
Le 04/09/2015 04:30, Chris Angelico a écrit : On Fri, Sep 4, 2015 at 12:17 PM, Vincent Vande Vyvre wrote: import urllib.request urllib.request.urlopen('http://example.org') Thanks, that works with 3.4.0. No with 3.2.3 Hmm, not sure why it wouldn't. According to the docs [1] it should be avai

Re: No request in module urllib ?

2015-09-03 Thread MRAB
"license" for more information. import urllib urllib.request.urlopen('http://example.org') Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'request' Same error with Python 3.4.0 With packa

Re: No request in module urllib ?

2015-09-03 Thread Chris Angelico
On Fri, Sep 4, 2015 at 12:17 PM, Vincent Vande Vyvre wrote: >> import urllib.request >> urllib.request.urlopen('http://example.org') >> > > Thanks, that works with 3.4.0. No with 3.2.3 Hmm, not sure why it wouldn't. According to the docs [1] it should be available. But I don't have a 3.2 anywhere

Re: No request in module urllib ?

2015-09-03 Thread Vincent Vande Vyvre
Le 04/09/2015 04:08, Chris Angelico a écrit : On Fri, Sep 4, 2015 at 11:56 AM, Vincent Vande Vyvre wrote: Python 3.2.3 (default, Jun 18 2015, 21:46:42) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. im

Re: No request in module urllib ?

2015-09-03 Thread Chris Angelico
On Fri, Sep 4, 2015 at 11:56 AM, Vincent Vande Vyvre wrote: > Python 3.2.3 (default, Jun 18 2015, 21:46:42) > [GCC 4.6.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import urllib >&g

No request in module urllib ?

2015-09-03 Thread Vincent Vande Vyvre
Hi, Python 3.2.3 (default, Jun 18 2015, 21:46:42) [GCC 4.6.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import urllib >>> urllib.request.urlopen('http://example.org') Traceback (mo

Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Chris Angelico
On Fri, Jun 20, 2014 at 12:19 AM, Robin Becker wrote: > in practice [monkeypatching socket] worked well with urllib in python27. Excellent! That's empirical evidence of success, then. Like with all monkey-patching, you need to keep it as visible as possible, but if your driver script i

Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Robin Becker
t sure why it hasn't been updated beyond that), pull up urllib2.py, and step through manually, seeing where the hostname gets turned into an IP address. Hence, this code: . in practice this approach worked well with urllib in python27. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list

Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Chris Angelico
On Thu, Jun 19, 2014 at 9:51 PM, Robin Becker wrote: >> Since you mention urllib2, I'm assuming this is Python 2.x, not 3.x. >> The exact version may be significant. >> > I can use python >= 3.3 if required. The main reason I ask is in case something's changed. Basically, what I did was go to my

Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Robin Becker
.. Since you mention urllib2, I'm assuming this is Python 2.x, not 3.x. The exact version may be significant. I can use python >= 3.3 if required. Can you simply query the server by IP address rather than host name? According to the docs, urllib2.urlopen() doesn't check the certific

Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Chris Angelico
On Thu, Jun 19, 2014 at 7:22 PM, Robin Becker wrote: > I want to run torture tests against an https server on domain A; I have > configured apache on the server to respond to a specific hostname ipaddress. > > I don't want to torture the live server so I have set up an alternate > instance on a di

urllib/urllib2 support for specifying ip address

2014-06-19 Thread Robin Becker
I want to run torture tests against an https server on domain A; I have configured apache on the server to respond to a specific hostname ipaddress. I don't want to torture the live server so I have set up an alternate instance on a different ip address. Is there a way to get urlib or urllib2

urllib with x509 certs

2014-05-16 Thread Luis Miguel Rojas Aguilera
Hi, i tried what you suggest but still asking me for the password, this time twice. Please i need help so this is for my thesis. VII Escuela Internacional de Verano en la UCI del 30 de junio al 11 de julio de 2014. Ver www.uci.cu -- https://mail.python.org/mailman/listinfo/python-list

Re: Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Roy Smith
In article , Marcus wrote: > I'm trying to use urllib and urllib2 to open an url + login_data in a for > loop. Step 1: Ignore all that crap and get http://www.python-requests.org/ > How can I display when successfully logged in and how to show when the > login is denied?

Re: Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Marcus
It's not that hard to find a program that does this already. But I'm trying to learn how to use these modules to create this. I've started it and now i want to complete it so I can create another program and learn more about other stuff, maybe a Twitter script or something. How do I learn when n

Re: Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Chris Angelico
On Mon, Mar 3, 2014 at 11:51 PM, Marcus wrote: > Yes, it's only for my own use on my local WordPress installation. Only > educational use. What are you trying to learn, exactly? How to break into a WP site? Still dubious. ChrisA -- https://mail.python.org/mailman/listinfo/python-list

Re: Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Marcus
Yes, it's only for my own use on my local WordPress installation. Only educational use. -- https://mail.python.org/mailman/listinfo/python-list

Re: Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Chris Angelico
On Mon, Mar 3, 2014 at 11:35 PM, Marcus wrote: > This is the code right now: http://pastebin.com/pE1YZX2K That looks short enough to include in-line, no need to point us to an external site :) So basically, you're doing a dictionary attack. May I ask why you're doing this, exactly? ChrisA -- h

Re: Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Chris Angelico
On Mon, Mar 3, 2014 at 11:44 PM, Chris Angelico wrote: > So basically, you're doing a dictionary attack. May I ask why you're > doing this, exactly? oops, misclicked. I note that the user name 'alex' does not appear to match your name. I'm going to want a good reason for this code to be written,

Re: Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Chris Angelico
On Mon, Mar 3, 2014 at 11:44 PM, Chris Angelico wrote: > So basically, you're doing a dictionary attack. May I ask why you're > doing this, exactly? -- https://mail.python.org/mailman/listinfo/python-list

Re: Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Marcus
This is the code right now: http://pastebin.com/pE1YZX2K -- https://mail.python.org/mailman/listinfo/python-list

Iterate through a list and try log in to a website with urllib and re

2014-03-03 Thread Marcus
Hello, I'm trying to use urllib and urllib2 to open an url + login_data in a for loop. How can I display when successfully logged in and how to show when the login is denied? I've tried use this: html_content = urllib2.urlopen(url).read() re.findall('ERROR: The password you

Re: Python: 404 Error when trying to login a webpage by using 'urllib' and 'HTTPCookieProcessor'

2014-01-12 Thread Terry Reedy
On 1/12/2014 7:17 AM, KMeans Algorithm wrote: But I get a "404" error (Not Found). The page "https://www.mysite.com/loginpage"; does exist Firefox tells me the same thing. If that is a phony address, you should have said so. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/

Re: Python: 404 Error when trying to login a webpage by using 'urllib' and 'HTTPCookieProcessor'

2014-01-12 Thread xDog Walker
'd > be ideal if it exists, though. I think you can set debug on httplib before using urllib to get the header traffic printed. I don't recall exactly how to do it though. -- Yonder nor sorghum stenches shut ladle gulls stopper torque wet strainers. -- https://mail.python.org/mailman/listinfo/python-list

Re: Python: 404 Error when trying to login a webpage by using 'urllib' and 'HTTPCookieProcessor'

2014-01-12 Thread Chris Angelico
On Sun, Jan 12, 2014 at 11:17 PM, KMeans Algorithm wrote: > The page "https://www.mysite.com/loginpage"; does exist PS. If it's not an intranet site and the URL isn't secret, it'd help if we could actually try things out. One of the tricks I like to use is to access the same page with a different

Re: Python: 404 Error when trying to login a webpage by using 'urllib' and 'HTTPCookieProcessor'

2014-01-12 Thread Chris Angelico
On Sun, Jan 12, 2014 at 11:17 PM, KMeans Algorithm wrote: > What am I doing wrong? Thank you very much. I can't say what's actually wrong, but I have a few ideas for getting more information out of the system... > opener = urllib2.build_opener(urllib2.HTTPCookieProcessor()) You don't do anythin

Python: 404 Error when trying to login a webpage by using 'urllib' and 'HTTPCookieProcessor'

2014-01-12 Thread KMeans Algorithm
I'm trying to log in a webpage by using 'urllib' and this piece of code - import urllib2,urllib,os opener = urllib2.build_opener(urllib2.HTTPCookieProcessor()) login = urllib.urlencode({'username':'john', 'password':'foo'}) url = &q

Re: urllib and authentication script integration

2013-12-23 Thread Chris Angelico
formation would be helpful. I gather from your subject line that you're using urllib; that's a start. What version of Python, and what platform(s)? What sort of authentication are you trying to do? Can you cut down the code in question until it's no longer sensitive (that is, so it

RE: urllib and authentication script integration

2013-12-23 Thread Jeff James
I have some simple code I would like to share with someone that can assist me in integrating authentication script into. I'm sure it's an easy answer for any of you. I am still researching, but on this particular project, time is of the essence and this is the only missing piece of the puzzle for

Re: Use of urllib (update)

2013-12-18 Thread Joel Goldstick
On Wed, Dec 18, 2013 at 10:16 AM, Jeff James wrote: > Folks, I promise I'll get to the point where my questions aren't so basic, > but I'm just now starting to get into Python. So I'm using the urllib > script to check to make sure our company sites are up. As s

RE: Use of urllib (update)

2013-12-18 Thread Jeff James
Folks, I promise I'll get to the point where my questions aren't so basic, but I'm just now starting to get into Python. So I'm using the urllib script to check to make sure our company sites are up. As stated earlier, I have three sites which require some form of authen

Re: Question RE urllib

2013-12-17 Thread Tobiah
ould be including in the script ? There are about 30 or 40 sites that I have listed in all. I just use those in the following script as examples. Thanks import urllib sites = ["http://www.amazon.com/";, "https://internalsite.com/intranet.html";, etc.] for sit

Re: Question RE urllib

2013-12-17 Thread Larry Martell
es that I have listed in all. I > just use those in the following script as examples. Thanks > > import urllib > > sites = ["http://www.amazon.com/";, "https://internalsite.com/intranet.html";, > etc.] > > for site in sites: > try: > url

RE: Question RE urllib

2013-12-17 Thread Jeff James
cked internally ? Only one of the sites reporting as down is "https" but all are internal sites. Is there some other component I should be including in the script ? There are about 30 or 40 sites that I have listed in all. I just use those in the following script as examples. Tha

Re: Question RE urllib

2013-12-16 Thread William Ray Wing
hat I have listed in all. I just use those in the > following script as examples. Thanks > > import urllib > > sites = ["http://www.amazon.com/";, "https://internalsite.com/intranet.html";, > etc.] > > for site in sites: > try: >

Re: Question RE urllib

2013-12-16 Thread John Gordon
In Jeff James writes: > --f46d04479f936227ee04edac31bd > Content-Type: text/plain; charset=ISO-8859-1 > Sorry to be a pain here, guys, as I'm also a newbie at this as well. > Where, exactly in the script would I place the " print str(e) " ? except Exception, e: print site + " is

Re: Question Re urllib (Resolved)

2013-12-16 Thread Jeff James
ttps" but all are internal > sites. Is there some other component I should be including in the script ? > There are about 30 or 40 sites that I have listed in all. I just use those > in the following script as examples. Thanks > > import urllib > > sites = ["http://

Re: Question RE urllib

2013-12-16 Thread Larry Martell
order for the >> > home >> >> > page to come up. Could this be due to some port being blocked >> > internally ? >> > Only one of the sites reporting as down is "https" but all are internal >> > sites. Is there some other component I s

RE: Question RE urllib

2013-12-16 Thread Jeff James
ng blocked > internally ? > > Only one of the sites reporting as down is "https" but all are internal > > sites. Is there some other component I should be including in the > script ? > > There are about 30 or 40 sites that I have listed in all. I just use > those

Re: Question Re urllib (Jeff James)

2013-12-16 Thread Larry Martell
t; > Cc: python-list@python.org 'python-list@python.org');> > Date: Mon, 16 Dec 2013 06:54:48 -0500 > Subject: Re: Question RE urllib > On Mon, Dec 16, 2013 at 6:40 AM, Jeff James > > > wrote: > > So I'm using the following script to check our sites to ma

Re: Question Re urllib (Jeff James)

2013-12-16 Thread Jeff James
/my..com/intranet.html* is down* http://#.main..com/psso/pssignsso.asp?dbname=FSPRD90 * is down* http://sharepoint..com/regions/west/PHX_NSC/default.aspx * is down* Cc: python-list@python.org Date: Mon, 16 Dec 2013 06:54:48 -0500 Subject: Re: Question RE urllib On Mon, Dec 16, 2013

Re: Question RE urllib

2013-12-16 Thread Tim Chase
On 2013-12-16 04:40, Jeff James wrote: > These sites do not require a logon in order for the home > page to come up. Could this be due to some port being blocked > internally ? Only one of the sites reporting as down is "https" but > all are internal sites. Is there some other component I should

Re: Question RE urllib

2013-12-16 Thread Larry Martell
t I have listed in all. I just use those > in the following script as examples. Thanks > > import urllib > > sites = ["http://www.amazon.com/";, "https://internalsite.com/intranet.html";, > etc.] > > for site in sites: > try: > urllib

Question RE urllib

2013-12-16 Thread Jeff James
cked internally ? Only one of the sites reporting as down is "https" but all are internal sites. Is there some other component I should be including in the script ? There are about 30 or 40 sites that I have listed in all. I just use those in the following script as examples. Tha

Re: Help regarding urllib

2013-08-24 Thread Dave Angel
malhar vora wrote: > On Saturday, August 24, 2013 4:15:01 PM UTC+5:30, malhar vora wrote: >> Hello All, >> >> >> >> >> >> I am simply fetching data from robots.txt of a url. Below is my code. >> >> >> >> siteurl = siteurl.rstrip("/") > > Sorry for last complete. It was sent by mistake. > >

Re: Help regarding urllib

2013-08-24 Thread malhar vora
On Saturday, August 24, 2013 4:15:01 PM UTC+5:30, malhar vora wrote: > Hello All, > > > > > > I am simply fetching data from robots.txt of a url. Below is my code. > > > > siteurl = siteurl.rstrip("/") Sorry for last complete. It was sent by mistake. Here is my code. siteurl = siteurl.rs

Help regarding urllib

2013-08-24 Thread malhar vora
Hello All, I am simply fetching data from robots.txt of a url. Below is my code. siteurl = siteurl.rstrip("/") -- http://mail.python.org/mailman/listinfo/python-list

Re: Must we include urllib just to decode a URL-encoded string, when using Requests?

2013-06-13 Thread Burak Arslan
On 06/13/13 16:25, Dotan Cohen wrote: On Thu, Jun 13, 2013 at 4:20 PM, Robert Kern wrote: Yes. Do you think there is a problem with doing so? I'm pretty sure that Requests will use either urllib or urllib2, depending on what is available on the server. I would like to use whatever Req

Re: Must we include urllib just to decode a URL-encoded string, when using Requests?

2013-06-13 Thread Dotan Cohen
) - p): > print(m) > Thank you. Python is a beautiful language, I cannot believe that the set(sys.modules)-p line does what it does! Interestingly, on my system with Python3 neither urllib nor urllib2 are imported, only urllib3 which I had not heard of until now: __future__ _json ate

Re: Must we include urllib just to decode a URL-encoded string, when using Requests?

2013-06-13 Thread Dotan Cohen
On Thu, Jun 13, 2013 at 4:34 PM, Robert Kern wrote: >> I'm pretty sure that Requests will use either urllib or urllib2, >> depending on what is available on the server. > > No, it doesn't. It gets its quote() function from urllib always. > I see, thanks. Then

Re: Must we include urllib just to decode a URL-encoded string, when using Requests?

2013-06-13 Thread Robert Kern
On 2013-06-13 14:25, Dotan Cohen wrote: On Thu, Jun 13, 2013 at 4:20 PM, Robert Kern wrote: Yes. Do you think there is a problem with doing so? I'm pretty sure that Requests will use either urllib or urllib2, depending on what is available on the server. No, it doesn't. It gets

Re: Must we include urllib just to decode a URL-encoded string, when using Requests?

2013-06-13 Thread Dotan Cohen
On Thu, Jun 13, 2013 at 4:20 PM, Robert Kern wrote: > Yes. Do you think there is a problem with doing so? > I'm pretty sure that Requests will use either urllib or urllib2, depending on what is available on the server. I would like to use whatever Requests is currently using, rather

Re: Must we include urllib just to decode a URL-encoded string, when using Requests?

2013-06-13 Thread Robert Kern
On 2013-06-13 14:05, Dotan Cohen wrote: I am using the Requests module to access remote URLs. Sometimes I need to URL-decode or URL-encode a string (via RFC 3986). Must I import urllib or urllib2 just to use their quote() and unquote() methods? Yes. Do you think there is a problem with doing

Must we include urllib just to decode a URL-encoded string, when using Requests?

2013-06-13 Thread Dotan Cohen
I am using the Requests module to access remote URLs. Sometimes I need to URL-decode or URL-encode a string (via RFC 3986). Must I import urllib or urllib2 just to use their quote() and unquote() methods? Does not Requests have such an ability, and perhaps I just cannot find it? On Stack Overflow

RE: newbie question about confusing exception handling in urllib

2013-04-12 Thread Prasad, Ramit
Steven D'Aprano wrote: > try: > main() > except Exception as err: > log(err) > print("Sorry, an unexpected error has occurred.") > print("Please contact support for assistance.") > sys.exit(-1) > > I like the traceback[0] module for logging last exception thrown. See tracebac

Re: newbie question about confusing exception handling in urllib

2013-04-09 Thread Ian Kelly
On Tue, Apr 9, 2013 at 5:41 AM, wrote: > try: > response = urllib.request.urlopen(request) > content = response.read() > except BaseException as ue: > if (isinstance(ue, socket.timeout) or (hasattr(ue, "reason") and > isinstance(ue.reason, sock

Re: newbie question about confusing exception handling in urllib

2013-04-09 Thread Chris Angelico
On Wed, Apr 10, 2013 at 1:05 AM, Steven D'Aprano wrote: > One exception to this rule (no pun intended) is that sometimes you want > to hide the details of unexpected tracebacks from your users. In that > case, it may be acceptable to wrap your application's main function in a > try block, catch an

Re: newbie question about confusing exception handling in urllib

2013-04-09 Thread Steven D'Aprano
On Tue, 09 Apr 2013 06:19:09 -0700, cabbar wrote: > How do I > handle all other exceptions, just say Exception: and handle them? I want > to silently ignore them. Please don't. That is normally poor practice, since it simply hides bugs in your code. As a general rule, you should only catch exce

Re: newbie question about confusing exception handling in urllib

2013-04-09 Thread Peter Otten
cab...@gmail.com wrote: > Ah, looks better. > > But, 2 questions: > > 1. I should also catch ConnectionResetError I am guessing. Does it need a special reaction? If so give it its own except suite. > 2. How do I handle all other exceptions, just say Exception: and handle > them? I want to sile

Re: newbie question about confusing exception handling in urllib

2013-04-09 Thread Terry Jan Reedy
urllib and in python in general... Basically, what I want to do is very simple, Very funny ;-). What you are trying to do, as your first project, is interact with the large, multi-layered, non=deterministic monster known as Internet, with timeout handling, through multiple layers of library code

Re: newbie question about confusing exception handling in urllib

2013-04-09 Thread cabbar
Hi, > > > > I have been using Java/Perl professionally for many years and have been > trying to learn python3 recently. As my first program, I tried writing a > class for a small project, and I am having really hard time understanding > exception handling in urllib

Re: newbie question about confusing exception handling in urllib

2013-04-09 Thread Peter Otten
cab...@gmail.com wrote: > Hi, > > I have been using Java/Perl professionally for many years and have been > trying to learn python3 recently. As my first program, I tried writing a > class for a small project, and I am having really hard time understanding > exception handlin

newbie question about confusing exception handling in urllib

2013-04-09 Thread cabbar
Hi, I have been using Java/Perl professionally for many years and have been trying to learn python3 recently. As my first program, I tried writing a class for a small project, and I am having really hard time understanding exception handling in urllib and in python in general... Basically

Re: using urllib on a more complex site

2013-02-24 Thread Adam W.
; I don't get the all the source I need, its just the navigation buttons.  Now > I assume they are using some CSS/javascript witchcraft to load all the useful > data later, so my question is how do I make urllib "wait" and grab that data > as well? > > > > >

Re: using urllib on a more complex site

2013-02-24 Thread Adam W.
> > > > > I don't get the all the source I need, its just the navigation buttons. > > Now I assume they are using some CSS/javascript witchcraft to load all the > > useful data later, so my question is how do I make urllib "wait" and grab > > that

Re: using urllib on a more complex site

2013-02-24 Thread Dave Angel
question is how do I make urllib "wait" and grab that data as well? The CSS and the jpegs, and many other aspects of a web "page" are loaded explicitly, by the browser, when parsing the tags of the page you downloaded. There is no sooner or later. The website won't

Re: using urllib on a more complex site

2013-02-24 Thread Chris Rebert
avascript witchcraft to load all the > useful data later, so my question is how do I make urllib "wait" and grab > that data as well? > urllib isn't a web browser. It just requests the single (in this case, HTML) file from the given URL. It does not parse the HTML (indeed,

using urllib on a more complex site

2013-02-24 Thread Adam W.
w.vudu.com/movies/#tag/99centOfTheDay/99c%20Rental%20of%20the%20day').read() I don't get the all the source I need, its just the navigation buttons. Now I assume they are using some CSS/javascript witchcraft to load all the useful data later, so my question is how do I make urllib "

Re: Python 2.7 import socket urllib fails to load, module not found

2012-08-17 Thread Christian Heimes
Am 17.08.2012 21:20, schrieb wdt...@comcast.net: > Just installed python 2.7 and using with web2py. > > When running python from command line to bring up web2py server, get errors > that python socket and urllib modules cannot be found, can't be loaded. This > is not a w

Re: Python 2.7 import socket urllib fails to load, module not found

2012-08-17 Thread Emile van Sebille
On 8/17/2012 2:22 PM wdt...@comcast.net said... Done - tail end of the python path had a missing bit...gr... thanks so much Well it's bizarre - now it doesn't. did an import sys from within interpreter, then did import socket. Worked the first time. Restarted and it happened again. Th

Re: Python 2.7 import socket urllib fails to load, module not found

2012-08-17 Thread wdtate
On Friday, August 17, 2012 3:20:48 PM UTC-4, (unknown) wrote: > Just installed python 2.7 and using with web2py. > > > > When running python from command line to bring up web2py server, get errors > that python socket and urllib modules cannot be found, can't be l

Re: Python 2.7 import socket urllib fails to load, module not found

2012-08-17 Thread wdtate
On Friday, August 17, 2012 5:15:35 PM UTC-4, (unknown) wrote: > > > > > So, try the following in both environments: > > > import sys > > > for ii in sys.path: print ii > > > > > > You'll likely find diffferences between the two. > > > > > In the pythonwin environment, try: > > >

Re: Python 2.7 import socket urllib fails to load, module not found

2012-08-17 Thread wdtate
> > So, try the following in both environments: > import sys > for ii in sys.path: print ii > > You'll likely find diffferences between the two. > In the pythonwin environment, try: > > > > import socket > > print socket.__file__ > > > Chances are the __file__'s director

  1   2   3   4   5   >