Thanks Massimo, I'll try this when I get home. I've sent you one of the links I'm talking about (can't post it publicly).
Much appreciated. LB On Nov 4, 4:51 pm, mdipierro <mdipie...@cs.depaul.edu> wrote: > Can you post one of these links so we can look at it in particular? > I think this may help: > > http://stackoverflow.com/questions/301924/python-urllib-urllib2-httpl... > > On Nov 4, 10:23 am, LB22 <latn.bl...@googlemail.com> wrote: > > > > > Hi, > > > OK to be a little less ambiguous (apologies for that), the redirects > > of the urls I'm probing happen within scripts, like those used in > > affiliate links, or url shorteners. But it just so happens that when > > say a user clicks a link, they don't only get redirected from point A > > to point B, but actually get sent through point A.1, point A.2, and > > then point A.3. > > > The purpose of this is to gather the domains to determine which ones > > to add to a whitelist. > > > Thanks > > > On Nov 4, 4:05 pm, mdipierro <mdipie...@cs.depaul.edu> wrote: > > > > what do you mean by "masked". Do you have to parse pages or is this > > > done via HTTP 303/307? > > > > On Nov 4, 9:54 am, LB22 <latn.bl...@googlemail.com> wrote: > > > > > Hi Everyone, > > > > > I'm not sure this is strictly a web2py question, but I figured I'd > > > > pose the question anyway, so here it goes. > > > > > I had a task of going through several hundred urls to find out where > > > > they lead, so I used URLLIB2 in a web2py app to get and display the > > > > end destinations of the "masked" links. This part was easy. > > > > > The problem is that many of the links hop through more than one URL > > > > before reaching the destination. So now I'm trying to retrieve the > > > > URLs that are in the redirect chain. I'm not quite sure how to do > > > > this. > > > > > In a nutshell, I want to get each requested URL in the journey to the > > > > end URL, given the start URL. > > > > > Does anyone know how to achieve this? > > > > > Thanks > > > > > LB22 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to web2py@googlegroups.com To unsubscribe from this group, send email to web2py+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/web2py?hl=en -~----------~----~----~----~------~----~------~--~---
