The following solution works:
I set my user-agent to:
VerseOfTheDaySitemapRobot/1.0 (http://www.TheVerseOfTheDay.info)
By doing:
ini_set('user_agent', "VerseOfTheDaySitemapRobot/1.0
(http://www.TheVerseOfTheDay.info)");
When ran by a cron job this causes AWSTATS to treat the hits as:
Unknown r
On Tue, 2010-11-30 at 00:19 -0200, Thiago H. Pojda wrote:
> Quit top posting.
>
> On Mon, Nov 29, 2010 at 9:55 PM, Ron Piggott > wrote:
>
> >
> > My issue with the user agent is unresolved. I need to do more research to
> > see how AWSTATS distinguishes between a robot crawling the site and a
Quit top posting.
On Mon, Nov 29, 2010 at 9:55 PM, Ron Piggott wrote:
>
> My issue with the user agent is unresolved. I need to do more research to
> see how AWSTATS distinguishes between a robot crawling the site and a web
> page user and set the user-agent accordingly.
>
Ron,
AWSTATS probab
: Shreyas Agasthya
Sent: Monday, November 29, 2010 1:35 AM
To: Ron Piggott
Cc: PHP General List
Subject: Re: [PHP] Fw: Spoofing user_agent
Ron,
Can you let us know if this whole thing that you were trying to do worked? I
see that very few actually bring a thread to a logical conclusion either by
> On 25 November 2010 15:38, Ron Piggott wrote:
>>
>> Is "User Agent" suppose to have a hyphen "-" ? Ron
> http://en.wikipedia.org/wiki/User_agent "... the identity is
> transmitted via the User-Agent request header, ... "
http://tools.ietf.org/html/rfc1945#section-10.15
User-Agent
Message- From: Richard Quadling
> Sent: Thursday, November 25, 2010 9:16 AM
> To: Deva
> Cc: Shreyas Agasthya ; Ron Piggott ; php-general@lists.php.net ;
> a...@ashleysheridan.co.uk
> Subject: Re: [PHP] Fw: Spoofing user_agent
>
> On 25 November 2010 11:32, Deva wrote:
>
iggott ; php-general@lists.php.net ;
a...@ashleysheridan.co.uk
Subject: Re: [PHP] Fw: Spoofing user_agent
On 25 November 2010 11:32, Deva wrote:
Use curl
http://php.net/manual/en/book.curl.php
On Thu, Nov 25, 2010 at 4:41 PM, Shreyas Agasthya
wrote:
I feel you should use more of the 4th me
7;);
>> > $url = "http://www.example.com";; <http://www.example.com%22;>
>> >
>> > $input = file_get_contents($url);
>> >
>> >
>> >
>> > The Verse of the Day
>> > “Encouragement from God’s Word”
>> > htt
> >
> >
> >
> > The Verse of the Day
> > “Encouragement from God’s Word”
> > http://www.TheVerseOfTheDay.info
> >
> > *From:* Shreyas Agasthya
> > *Sent:* Thursday, November 25, 2010 4:21 AM
> > *To:* Ron Piggott
> > *Cc:* php-general
$input = file_get_contents($url);
>
>
>
> The Verse of the Day
> “Encouragement from God’s Word”
> http://www.TheVerseOfTheDay.info
>
> *From:* Shreyas Agasthya
> *Sent:* Thursday, November 25, 2010 4:21 AM
> *To:* Ron Piggott
> *Cc:* php-general@lists.php.n
od’s Word”
http://www.TheVerseOfTheDay.info
From: Shreyas Agasthya
Sent: Thursday, November 25, 2010 4:21 AM
To: Ron Piggott
Cc: php-general@lists.php.net ; a...@ashleysheridan.co.uk
Subject: Re: [PHP] Fw: Spoofing user_agent
A standard HTTP Request headers is : User Agent (without the underscore).
--S
Thanks. Ron
The Verse of the Day
“Encouragement from God’s Word”
http://www.TheVerseOfTheDay.info
From: Shreyas Agasthya
Sent: Thursday, November 25, 2010 4:21 AM
To: Ron Piggott
Cc: php-general@lists.php.net ; a...@ashleysheridan.co.uk
Subject: Re: [PHP] Fw: Spoofing user_agent
A standard
Ron
>
> The Verse of the Day
> “Encouragement from God’s Word”
> http://www.TheVerseOfTheDay.info
>
> From: a...@ashleysheridan.co.uk
> Sent: Thursday, November 25, 2010 3:34 AM
> To: Ron Piggott ; php-general@lists.php.net
> Subject: Re: [PHP] Fw: Spoofing user_agent
>
> You nee
gott ; php-general@lists.php.net
Subject: Re: [PHP] Fw: Spoofing user_agent
You need to set it in the header request you make. Putting it in the script
you're using as a spider with ini_set won't do anything because the Target site
doesn't know anything about it.
Thanks,
Ash
http:/
ott"
Date: Thu, Nov 25, 2010 08:25
Subject: [PHP] Fw: Spoofing user_agent
To:
I have wrote a script to generate a sitemap of my web site. It crawls all of
the site web pages. (About 30,000)
I need help to spoof the user_agent variable so the stats program running in
the background ( “
I have wrote a script to generate a sitemap of my web site. It crawls all of
the site web pages. (About 30,000)
I need help to spoof the user_agent variable so the stats program running in
the background ( “AWSTATS” ) will treat the crawl as a bot, not browsing usage.
The sitemap generat
16 matches
Mail list logo
