Thanks Barry for the great explanation. Then I wonder if instead of blocking when it is detected but to send a big splash page as the response for all requests with the correct link and a message. I guess like advertising.
On Wed, Jan 26, 2011 at 10:21 AM, Barry Hunter <[email protected]>wrote: > They might even be doing it unintentionally. > > The number of 'open proxies' on AppEngine - is very high. If someone > uses said proxy against your site, the link can end up in a search > engine, and because the proxy rewrite links, search engine crawlers > get a whole new site to crawl. > - ideally the proxy should be blocking crawlers - at the very least. > (and in a perverse way because the proxy site hosts content from many > domains (ie subjects) - it can end up getting quite a diverse > portfolio of content) > > > The intentional ones, are doing to to increase exposure to their site > mainly. They can put AdSence on the pages (even just rewriting the > adsence it of the original content producer) - or similar advertising > to raise money. > - basically they steal trafffic from your site, by stealing the > content - ie very little work. > > > (speculating on Kyle's case, as we dont know the specifics ;) > > > On 26 January 2011 17:01, Stephen Johnson <[email protected]> wrote: > > This issue has surfaced before. What is the purpose behind them doing > this? > > (sorry if that's a stupid question) > > > > On Wed, Jan 26, 2011 at 7:05 AM, Barry Hunter <[email protected]> > > wrote: > >> > >> Doesnt help you for the results already there (although it should help > >> to make them disappear with time) - and its a cat and mouse game if > >> they try changing appid. > >> > >> but you can block them from reading your content. I guess they must be > >> using URLFetch to grab your content. The User-Agent on such calls > >> include the appid of the calling app - just selectively block on that > >> ;) > >> > >> (and yes I know it doesnt answer your actual question) > >> > >> > >> > >> On 26 January 2011 12:59, Kyle Jensen <[email protected]> wrote: > >> > Hi - > >> > > >> > How can I find out who owns a particular application? > >> > > >> > The app does not use a custom domain, so DNS/registration info is no > >> > help. I noticed a app that is mirroring all my content and is showing > >> > up next to me in search results. So, clearly, I'd like to ask them to > >> > stop doing that. > >> > > >> > Long time GAE user, Kyle > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups "Google App Engine" group. > >> > To post to this group, send email to > [email protected]. > >> > To unsubscribe from this group, send email to > >> > [email protected]<google-appengine%[email protected]> > . > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-appengine?hl=en. > >> > > >> > > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]<google-appengine%[email protected]> > . > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > . > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
