[us...@httpd] RE: Caching "virtual directory" requests

[Karim Zaki]

Hi all,

Any help with this request would be highly appreciated :).

Thanks,
Karim

From: Karim Zaki
Sent: Tuesday, December 30, 2008 3:10 PM
To: users@httpd.apache.org
Subject: Caching "virtual directory" requests

Hello everyone,

I'm running Apache 2.2.11 with mod_jk and mod_cache. Apache doesn't seem to be 
caching requests for "virtual directories" (i.e. requests for URIs that don't 
end in a file name). My back-end Tomcat application uses such URLs (e.g. 
http:///http://%3chost-%3e/%3cvirtual-directory>>) 
which result in a dynamic page being generated. Requests to such URIs come to 
Apache, they are forwarded to Tomcat using mod_jk, but the response is not 
cached. Is there a way to convince Apache to cache these URLs?

Thanks,
Karim

---
"Although the moon is smaller than the earth, it is farther away."

Re: [us...@httpd] RE: Caching "virtual directory" requests

[Eric Covener]

On Sun, Jan 4, 2009 at 5:40 AM, Karim Zaki  wrote:
> I'm running Apache 2.2.11 with mod_jk and mod_cache. Apache doesn't seem to
> be caching requests for "virtual directories" (i.e. requests for URIs that
> don't end in a file name). My back-end Tomcat application uses such URLs
> (e.g. http:///) which result in a dynamic page
> being generated. Requests to such URIs come to Apache, they are forwarded to
> Tomcat using mod_jk, but the response is not cached. Is there a way to
> convince Apache to cache these URLs?

I don't see why Apache would treat such a URL differently.

Have you checked the other cacheability rules in the caching guide and
turned on LogLevel debug?

-- 
Eric Covener
cove...@gmail.com

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

[us...@httpd] help

[John Winther]

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
  "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] help

[Eric Covener]

On Sat, Jan 3, 2009 at 10:29 AM, John Winther  wrote:
>

I couldn't have guessed that the Subject would be the highlight of this message.


-- 
Eric Covener
cove...@gmail.com

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] Unable to compile Apache-2.2.11 on Windows using Cygwin

[Morgan Gangwere]

William A. Rowe, Jr. wrote:

sathya sai wrote:
 
By looking at the difference in the source tree between Apache-2.2.9 &

Apache-2.2.11, I could see that "*.mak" & ".dep" are removed from the
latest source tree of Apache-2.2.11 & thus causing the problem.


nope - .mak/.dep files are added to the httpd-2.2.x-win32-src.zip package.

If you want a native build from the command line (sounds like you do) you
just need to grab the right package and ensure ms tools/sdk are in your
path.

FYI the other way 'round is to run srclib\apr\build\cvtdsp -2005
and then load apache.dsw into devenv (visual studio 2003+++).  Even
running the command line make when then run them through the console
devenv mode from the imported .sln / .vcproj files.

If you want a command line cyg/mingw build you don't do any of the above,
including using makefile.win.  You run ./configure and proceed as if you
were running in any other unix package.




...Also, why are you (a) compiling something in an odd way and (b) 
compiling under Cygwin for Windows when you'll get better perfomance on 
a standard win32 build.


--

Morgan Gangwere
Unknown Software
begin:vcard
fn:Morgan Gangwere
n:Gangwere;Morgan
org:Unknown Software;Programming
adr;dom:;;Contact for details.
email;internet:0.fracta...@gmail.com
title:Developer
tel;work:+1 505 514 2863
note;quoted-printable:Find me on IRC:=0D=0A=
	~indrora!fa...@irc.freenode.net=0d=0a=
	=0D=0A=
	via IM:=0D=0A=
	MSN: fractalsca...@passport.com=0d=0a=
	Yahoo: trogdor_t_b=0D=0A=
	GTalk (talk.google.com SSL): 0.fracta...@gmail.com=0d=0a=
	=0D=0A=
	I'm all over the net!
url:http://sonof.bandit.name/
version:2.1
end:vcard

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] Unsubscribe

[Morgan Gangwere]

Evan Platt wrote:

As the bottom of every message says:

To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org



how did this get mixed in by Threadr with the Tomcat discussion?

--

Morgan Gangwere
Unknown Software

begin:vcard
fn:Morgan Gangwere
n:Gangwere;Morgan
org:Unknown Software;Programming
adr;dom:;;Contact for details.
email;internet:0.fracta...@gmail.com
title:Developer
tel;work:+1 505 514 2863
note;quoted-printable:Find me on IRC:=0D=0A=
	~indrora!fa...@irc.freenode.net=0d=0a=
	=0D=0A=
	via IM:=0D=0A=
	MSN: fractalsca...@passport.com=0d=0a=
	Yahoo: trogdor_t_b=0D=0A=
	GTalk (talk.google.com SSL): 0.fracta...@gmail.com=0d=0a=
	=0D=0A=
	I'm all over the net!
url:http://sonof.bandit.name/
version:2.1
end:vcard


-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] Unsubscribe

[Evan Platt]

At 08:20 AM 1/4/2009, you wrote:

Evan Platt wrote:

As the bottom of every message says:
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org


how did this get mixed in by Threadr with the Tomcat discussion?


My guess is the person who wanted to unsubscribe picked a message, 
replied to it, then changed the subject to unsubscribe. 



-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
  "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] How to combine multiple set-cookie headers into one?

[Morgan Gangwere]

[snip]
A cookie has lots of parts. For one, it has a name. that name can be 
anything.

Second, it has a time-limit (experation date) in seconds until it dies.
Lastly, it has whatever content its going to have.

If you want to make sure the header is being sent add something like
x-test-header: foo

Here's a nice peice of documentation (albiet from a MSDN) on the HTTP 
Cookies.

http://msdn.microsoft.com/en-us/library/aa384321(VS.85).aspx
however, I'll paste the relavant block

-copypasta-
Cookie-Related Headers

There are two headers, Set-Cookie and Cookie, that are related to 
cookies. The Set-Cookie header is sent by the server in response to an 
HTTP request, which is used to create a cookie on the user's system. The 
Cookie header is included by the client application with an HTTP request 
sent to a server, if there is a cookie that has a matching domain and path.

Set-Cookie Header

The Set-Cookie response header uses the following format:

Copy Code

Set-Cookie: =[; =]...
[; expires=][; domain=]
[; path=][; secure][; httponly]

One or more string sequences (separated by semicolons) that follow the 
pattern name=value must be included in the Set-Cookie response header. 
The server can use these string sequences to store data on the client's 
system.


The expiration date is set by using the format expires=date, where date 
is the expiration date in Greenwich Mean Time (GMT). If the expiration 
date is not set, the cookie expires after the Internet session ends. 
Otherwise, the cookie is persisted in the cache until the expiration 
date. The date must follow the format DAY, DD-MMM-HH:MM:SS GMT, 
where DAY is the day of the week (Sun, Mon, Tue, Wed, Thu, Fri, Sat), DD 
is the day in the month (such as 01 for the first day of the month), MMM 
is the three-letter abbreviation for the month (Jan, Feb, Mar, Apr, May, 
Jun, Jul, Aug, Sep, Oct, Nov, Dec),  is the year, HH is the hour 
value in military time (22 would be 10:00 P.M., for example), MM is the 
minute value, and SS is the second value.


Specifying the domain name, using the pattern domain=domain_name, is 
optional for persistent cookies and is used to indicate the end of the 
domain for which the cookie is valid. Session cookies that specify a 
domain are rejected. If the specified domain name ending matches the 
request, the cookie tries to match the path to determine if the cookie 
should be sent. For example, if the domain name ending is 
.microsoft.com, requests to home.microsoft.com and support.microsoft.com 
would be checked to see if the specified pattern matches the request. 
The domain name must have at least two or three periods in it to prevent 
cookies from being set for widely used domain name endings, such as 
.com, .edu, and co.jp. Allowable domain names would be similar to 
.microsoft.com, .someschool.edu, and .someserver.co.jp. Only hosts 
within the specified domain can set a cookie for a domain.


Setting the path, using the pattern path=some_path, is optional and can 
be used to specify a subset of the URLs for which the cookie is valid. 
If a path is specified, the cookie is considered valid for any requests 
that match that path. For example, if the specified path is /example, 
requests with the paths /examplecode and /example/code.htm would match. 
If no path is specified, the path is assumed to be the path of the 
resource associated with the Set-Cookie header.


The cookie can also be marked as secure, which specifies that the cookie 
can be sent only to https servers.

-/copypasta-

I'd say that if you wished to test you could simply use the line
Set-Cookie: foo=bar;bar=baz;baz=bletch;domain=(domain?localhost)
and see if that works.

A good tool to look at the headers going by is the catch-all Wireshark, 
or the extention for Firefox, LiveHTTPHeaders. Wireshark is a Packet 
Sniffer, while LiveHTTPHeaders simply grabs the hooks for getting the 
headers from Firefox.


Dont have firefox? well then you should proabably get it. That and the 
Web Developer Toolbar (for Firefox)


--

Morgan Gangwere
Unknown Software
begin:vcard
fn:Morgan Gangwere
n:Gangwere;Morgan
org:Unknown Software;Programming
adr;dom:;;Contact for details.
email;internet:0.fracta...@gmail.com
title:Developer
tel;work:+1 505 514 2863
note;quoted-printable:Find me on IRC:=0D=0A=
	~indrora!fa...@irc.freenode.net=0d=0a=
	=0D=0A=
	via IM:=0D=0A=
	MSN: fractalsca...@passport.com=0d=0a=
	Yahoo: trogdor_t_b=0D=0A=
	GTalk (talk.google.com SSL): 0.fracta...@gmail.com=0d=0a=
	=0D=0A=
	I'm all over the net!
url:http://sonof.bandit.name/
version:2.1
end:vcard

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: use

Re: [us...@httpd] Unsubscribe

[Doug McNutt]

At 08:23 -0800 1/4/09, Evan Platt wrote:

At 08:20 AM 1/4/2009, you wrote:

how did this get mixed in by Threadr with the Tomcat discussion?


My guess is the person who wanted to unsubscribe picked a message, 
replied to it, then changed the subject to unsubscribe.




Right you are.  These headers were in the first Unsubscribe message.

Message-ID: 
In-Reply-To: <495e1e49.3090...@ice-sa.com>

A quick search for the message-id in the Reply-to header comes up with;

Message-ID: <495e1e49.3090...@ice-sa.com>
In-Reply-To: 
<1bca52cf5845e543b4b81aafef2afd7905f24...@lonsexc01.gta.travel.lcl>
Subject: Re: [us...@httpd] Isolating slow and fast connections using 
apache/modjk


Not all mail clients handle the In-Reply-To: header the same way and 
hardly any allow that chain to be exclusively used to keep threads 
associated with each other. It's a shame because of the Re: [users. 
.] AW: spelling corrections, and a bunch of other things that get 
"helpfully" added to Subject: lines.


But it's good netiquette to start a new thread with a fresh "new 
message". RFC-2822 is the place to start

--

--> A fair tax is one that you pay but I don't <--

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
  "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

[us...@httpd] help I do not understand

[swilting]

I have multiple domain names

renelacroute.fr
fakessh.eu

when I returned in the browser http://fakessh.eu

I find myself in the page index renelacroute.fr

when I returned in the browser http://www.fakessh.eu its ok return well
I do not understand

dig me back with the right answers

dig its ok return well

[r...@r13151 ~]# dig fakessh.eu any

; <<>> DiG 9.3.4-P1 <<>> fakessh.eu any
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 498
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 3

;; QUESTION SECTION:
;fakessh.eu.IN  ANY

;; ANSWER SECTION:
fakessh.eu. 38400   IN  MX  10 mail.fakessh.eu.
fakessh.eu. 38400   IN  A   87.98.186.232
fakessh.eu. 38400   IN  SOA r13151.ovh.net.
postmaster.fakessh.eu. 1230836225 10800 3600 604800 38400
fakessh.eu. 38400   IN  NS  dns12.ovh.net.
fakessh.eu. 38400   IN  NS  r13151.ovh.net.
fakessh.eu. 38400   IN  NS  ns.fakessh.eu.
fakessh.eu. 38400   IN  NS  ns12.ovh.net.

;; ADDITIONAL SECTION:
mail.fakessh.eu.38400   IN  A   87.98.186.232
ns12.ovh.net.   86392   IN  A   213.251.128.131
dns12.ovh.net.  86392   IN  A   213.251.188.131

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Jan  4 19:07:12 2009
;; MSG SIZE  rcvd: 244

[r...@r13151 ~]# 
[r...@r13151 ~]# dig www.fakessh.eu any

; <<>> DiG 9.3.4-P1 <<>> www.fakessh.eu any
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65347
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 2

;; QUESTION SECTION:
;www.fakessh.eu.IN  ANY

;; ANSWER SECTION:
www.fakessh.eu. 38400   IN  A   87.98.186.232

;; AUTHORITY SECTION:
fakessh.eu. 38400   IN  NS  ns12.ovh.net.
fakessh.eu. 38400   IN  NS  dns12.ovh.net.
fakessh.eu. 38400   IN  NS  r13151.ovh.net.
fakessh.eu. 38400   IN  NS  ns.fakessh.eu.

;; ADDITIONAL SECTION:
ns12.ovh.net.   86360   IN  A   213.251.128.131
dns12.ovh.net.  86360   IN  A   213.251.188.131

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Jan  4 19:07:44 2009
;; MSG SIZE  rcvd: 164

[r...@r13151 ~]# 





-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] help I do not understand

[Eric Covener]

On Sun, Jan 4, 2009 at 1:09 PM, swilting  wrote:
> I have multiple domain names
>
> renelacroute.fr
> fakessh.eu
>
> when I returned in the browser http://fakessh.eu
>
> I find myself in the page index renelacroute.fr
>
> when I returned in the browser http://www.fakessh.eu its ok return well

What does apache2ctl/apachectl -S say about your virtualhost configuration?

 The first listed vhost in a set of NameVirtualHosts is the default,
which might be why you see the "wrong" one.

http://httpd.apache.org/docs/2.2/vhosts/name-based.html

-- 
Eric Covener
cove...@gmail.com

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] Unsubscribe

[Evan Platt]

At 09:46 AM 1/4/2009, you wrote:

But it's good netiquette to start a new thread with a fresh "new 
message". RFC-2822 is the place to start


You're preaching to the choir. :-D 



-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
  "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] help I do not understand

[swilting]

I understand your answer but how to properly separate and that vhosts
does not mix

[r...@r13151 ~]# apachectl -S
VirtualHost configuration:
87.98.186.232:*is a NameVirtualHost
 default server www.renelacroute.fr
(/etc/httpd/conf/httpd.conf:993)
 port * namevhost www.renelacroute.fr
(/etc/httpd/conf/httpd.conf:993)
 port * namevhost www.fakessh.eu
(/etc/httpd/conf/httpd.conf:1003)
Syntax OK
[r...@r13151 ~]# 

I do not speak English well and the doc page I do not succeed in
translating

a way to transfer and cancel the default server
Le dimanche 04 janvier 2009 à 13:15 -0500, Eric Covener a écrit :
> On Sun, Jan 4, 2009 at 1:09 PM, swilting  wrote:
> > I have multiple domain names
> >
> > renelacroute.fr
> > fakessh.eu
> >
> > when I returned in the browser http://fakessh.eu
> >
> > I find myself in the page index renelacroute.fr
> >
> > when I returned in the browser http://www.fakessh.eu its ok return well
> 
> What does apache2ctl/apachectl -S say about your virtualhost configuration?
> 
>  The first listed vhost in a set of NameVirtualHosts is the default,
> which might be why you see the "wrong" one.
> 
> http://httpd.apache.org/docs/2.2/vhosts/name-based.html
> 



-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] help I do not understand

[swilting]

I understand your answer but how to properly separate and that vhosts
does not mix

[r...@r13151 ~]# apachectl -S
VirtualHost configuration:
87.98.186.232:*is a NameVirtualHost
 default server www.renelacroute.fr
(/etc/httpd/conf/httpd.conf:993)
 port * namevhost www.renelacroute.fr
(/etc/httpd/conf/httpd.conf:993)
 port * namevhost www.fakessh.eu
(/etc/httpd/conf/httpd.conf:1003)
Syntax OK
[r...@r13151 ~]# 

I do not speak English well and the doc page I do not succeed in
translating

a way to transfer and cancel the default server


Le dimanche 04 janvier 2009 à 13:15 -0500, Eric Covener a écrit :
> On Sun, Jan 4, 2009 at 1:09 PM, swilting  wrote:
> > I have multiple domain names
> >
> > renelacroute.fr
> > fakessh.eu
> >
> > when I returned in the browser http://fakessh.eu
> >
> > I find myself in the page index renelacroute.fr
> >
> > when I returned in the browser http://www.fakessh.eu its ok return well
> 
> What does apache2ctl/apachectl -S say about your virtualhost configuration?
> 
>  The first listed vhost in a set of NameVirtualHosts is the default,
> which might be why you see the "wrong" one.
> 
> http://httpd.apache.org/docs/2.2/vhosts/name-based.html
> 



-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] help I do not understand

[Eric Covener]

On Sun, Jan 4, 2009 at 2:44 PM, swilting  wrote:
> I understand your answer but how to properly separate and that vhosts
> does not mix
>
> [r...@r13151 ~]# apachectl -S
> VirtualHost configuration:
> 87.98.186.232:*is a NameVirtualHost
> default server www.renelacroute.fr
> (/etc/httpd/conf/httpd.conf:993)
> port * namevhost www.renelacroute.fr
> (/etc/httpd/conf/httpd.conf:993)
> port * namevhost www.fakessh.eu
> (/etc/httpd/conf/httpd.conf:1003)
> Syntax OK
> [r...@r13151 ~]#

Perhaps you just need:

ServerAlias fakessh.eu

In the virtualhost that currently has "ServerName www.fakessh.eu"


-- 
Eric Covener
cove...@gmail.com

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

[us...@httpd] Issues with mod_disk_cache and htcacheclean

[Neil Gunton]

I am using Apache 2.2.9 on Linux AMD64, built from source. There is one 
server running two builds of Apache - a lightweight front-end caching 
reverse proxy configuration using mod_disk_cache, and a heavyweight 
mod_perl back end. I use caching to relieve load on the server when many 
people request the same page at once. The website is dynamic and 
contains millions of page permutations. Thus the cache has a tendency to 
get fairly large, unless it is pruned. So I have been trying to use 
htcacheclean to achieve this. There have been some issues, which I will 
outline below.


First, I found that htcacheclean was not able to keep up with pruning 
the cache. It just kept growing. I initially ran htcacheclean in daemon 
mode, thus:


htcacheclean -i -t -n -d60 -p/var/cache/www -l1000M

CacheDirLevels was 3 and CacheDirLength 1.

The cache would just keep getting bigger, to multiple GB. Additionally, 
even doing a du on the cache could take hours to complete.


I also noticed that iowait would spike when I tried running htcacheclean 
in non-daemon mode. It would not keep up at all using the -n ("nice") 
option; when I took that off, the iowait would go through the roof and 
the process would take hours to complete. This was on a quad core AMD64 
server with 4 x 10k SCSI drives in hardware RAID0.


Upon investigation, I discovered that the cache was a lot deeper than I 
expected. In addition to the three levels specified in CacheDirLevels, 
there were then additional levels of subdirectories beneath ".vary" 
subdirs. For each .header file, there was a .vary subdir with three 
levels of directory below that. Simply traversing this tree with du 
could take a long time - hours sometimes, depending on how long the 
server had been running without a cache clear.


I discovered that the .vary subdirs were caused by my configuration, 
which was introducing a Vary http header. This came from two sources: 
First, mod_deflate. I found this out from this helpful page:


http://www.digitalsanctuary.com/tech-blog/general/apache-mod_deflate-and-mod_cache-issues.html

So I disabled mod_deflate, since it seemed to be producing a huge number 
of cache entries for each file - a different one for every browser. But 
after disabling mod_deflate, the .vary subdirs were still there. I also 
had this line in my config:


Header add Vary "Cookie"

This is necessary because users on my site set options for how the site 
is displayed. When I tried disabling this cookie Vary header, the number 
of directories went down substantially, to the expected three levels. 
The cache structure was much simpler, and it seemed that htcacheclean 
could keep up with this. However, the site was broken - since the same 
page for different users with different options would be cached only 
once. So someone who had "no ads" or "no pics" would request a page that 
someone else had recently requested (with different options), and they 
would get that other person's options. Not good. So I had to switch the 
vary header for cookies back on, so that pages would get differentiated 
in the cache based on cookie. But now I was back to square one - six 
effective levels of subdirectory, which htcacheclean could not keep up with.


After some thought, I ended up changing CacheDirLevels to 2, to try to 
reduce the depth of the tree. Now I had fewer subdirs, but more files in 
each one.


Also, the size of the cache, via du, always seems to be much higher than 
specified for htcacheclean. I lowered the limit to 100M, but still the 
cache is regularly up at 180MB or 200MB. This seems counter-intuitive, 
since htcacheclean doesn't appear to be taking the true size of the 
cache into account (i.e. including all the subdirs, which also take up 
space and presumably are what cause the discrepancy).


I also noticed something else: htcacheclean was leaving behind .header 
files. When it cleaned the .vary subdirs, it seemed to leave behind the 
corresponding .header files. These would accumulate, causing the iowait 
to gradually increase, presumably due to the size of the directories. I 
would rotate (clear) the cache manually at midnight. The behavior I 
would see (via munin monitoring tool) was that iowait would then remain 
at zero for about 12 hours, but then would gradually become visible as 
the number of .header files would accumulate.


So I wrote a perl script which could go through the cache, and look for 
.header files, and for each one found, see if a corresponding .vary 
subdir exists for it. If not, then the .header file is deleted. I then 
run another script to prune empty subdirectories. Currently I run this 
combination every 10 minutes - first a non-daemon invocation of 
htcacheclean, followed by the header prune script, followed by the empty 
subdirs pruning script. This seems to keep the cache small, and iowait 
is not noticeable any more, since the "junk" .header files are now 
disposed of regularly.


However, I'm not sure why I need to run this ki

Re: [us...@httpd] Issues with mod_disk_cache and htcacheclean

[William A. Rowe, Jr.]

Neil Gunton wrote:
> 
> I wasn't sure if I should post this on the dev list, since it seems to
> be more directed at the developers than other users. But the list
> guidelines said that "Configuration and support questions should be
> addressed to a user support group", and this seems to be that, so I'll
> post it here first.

Given the effort you took to put together your questions, I think the dev@
list would be very appropriate for developers to consider the current
approach and also consider changes in the upcoming httpd 2.4/3.0 release.

Please share this at d...@.

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] Issues with mod_disk_cache and htcacheclean

[Neil Gunton]

William A. Rowe, Jr. wrote:

Neil Gunton wrote:

I wasn't sure if I should post this on the dev list, since it seems to
be more directed at the developers than other users. But the list
guidelines said that "Configuration and support questions should be
addressed to a user support group", and this seems to be that, so I'll
post it here first.


Given the effort you took to put together your questions, I think the dev@
list would be very appropriate for developers to consider the current
approach and also consider changes in the upcoming httpd 2.4/3.0 release.

Please share this at d...@.


Thanks - I have done that.

Neil

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
  "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] How to combine multiple set-cookie headers into one?

[anson ho]

Hi Morgan,

Thanks for your reply.

The case is that, many cell phone clients can't handle multiple
Set-Cookie headers, and I don't have the right to change the the web
application to modify the reply headers. So, it seems to me that change the
headers in apache web server is the only option. I've read that
mod_header.so can change the headers, and I've tried but no success. So I
want to someone to point me to the correct direction. I am pretty new to
apache web server administration.

Anson Ho

On Mon, Jan 5, 2009 at 12:29 AM, Morgan Gangwere <0.fracta...@gmail.com>wrote:

> [snip]
> A cookie has lots of parts. For one, it has a name. that name can be
> anything.
> Second, it has a time-limit (experation date) in seconds until it dies.
> Lastly, it has whatever content its going to have.
>
> If you want to make sure the header is being sent add something like
> x-test-header: foo
>
> Here's a nice peice of documentation (albiet from a MSDN) on the HTTP
> Cookies.
> http://msdn.microsoft.com/en-us/library/aa384321(VS.85).aspx
> however, I'll paste the relavant block
>
>-copypasta-
> Cookie-Related Headers
>
> There are two headers, Set-Cookie and Cookie, that are related to cookies.
> The Set-Cookie header is sent by the server in response to an HTTP request,
> which is used to create a cookie on the user's system. The Cookie header is
> included by the client application with an HTTP request sent to a server, if
> there is a cookie that has a matching domain and path.
> Set-Cookie Header
>
> The Set-Cookie response header uses the following format:
>
> Copy Code
>
> Set-Cookie: =[; =]...
> [; expires=][; domain=]
> [; path=][; secure][; httponly]
>
> One or more string sequences (separated by semicolons) that follow the
> pattern name=value must be included in the Set-Cookie response header. The
> server can use these string sequences to store data on the client's system.
>
> The expiration date is set by using the format expires=date, where date is
> the expiration date in Greenwich Mean Time (GMT). If the expiration date is
> not set, the cookie expires after the Internet session ends. Otherwise, the
> cookie is persisted in the cache until the expiration date. The date must
> follow the format DAY, DD-MMM-HH:MM:SS GMT, where DAY is the day of the
> week (Sun, Mon, Tue, Wed, Thu, Fri, Sat), DD is the day in the month (such
> as 01 for the first day of the month), MMM is the three-letter abbreviation
> for the month (Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec),
>  is the year, HH is the hour value in military time (22 would be 10:00
> P.M., for example), MM is the minute value, and SS is the second value.
>
> Specifying the domain name, using the pattern domain=domain_name, is
> optional for persistent cookies and is used to indicate the end of the
> domain for which the cookie is valid. Session cookies that specify a domain
> are rejected. If the specified domain name ending matches the request, the
> cookie tries to match the path to determine if the cookie should be sent.
> For example, if the domain name ending is .microsoft.com, requests to
> home.microsoft.com and support.microsoft.com would be checked to see if
> the specified pattern matches the request. The domain name must have at
> least two or three periods in it to prevent cookies from being set for
> widely used domain name endings, such as .com, .edu, and co.jp. Allowable
> domain names would be similar to .microsoft.com, .someschool.edu, and .
> someserver.co.jp. Only hosts within the specified domain can set a cookie
> for a domain.
>
> Setting the path, using the pattern path=some_path, is optional and can be
> used to specify a subset of the URLs for which the cookie is valid. If a
> path is specified, the cookie is considered valid for any requests that
> match that path. For example, if the specified path is /example, requests
> with the paths /examplecode and /example/code.htm would match. If no path is
> specified, the path is assumed to be the path of the resource associated
> with the Set-Cookie header.
>
> The cookie can also be marked as secure, which specifies that the cookie
> can be sent only to https servers.
>-/copypasta-
>
> I'd say that if you wished to test you could simply use the line
> Set-Cookie: foo=bar;bar=baz;baz=bletch;domain=(domain?localhost)
> and see if that works.
>
> A good tool to look at the headers going by is the catch-all Wireshark, or
> the extention for Firefox, LiveHTTPHeaders. Wireshark is a Packet Sniffer,
> while LiveHTTPHeaders simply grabs the hooks for getting the headers from
> Firefox.
>
> Dont have firefox? well then you should proabably get it. That and the Web
> Developer Toolbar (for Firefox)
>
> --
>
> Morgan Gangwere
> Unknown Software
>
> -
> The official User-To-User support forum

Re: [us...@httpd] How to combine multiple set-cookie headers into one?

[Morgan Gangwere]

anson ho wrote:

Hi Morgan,
  
Thanks for your reply.


The case is that, many cell phone clients can't handle multiple 
Set-Cookie headers, and I don't have the right to change the the web 
application to modify the reply headers. So, it seems to me that change 
the headers in apache web server is the only option. I've read that 
mod_header.so can change the headers, and I've tried but no success. So 
I want to someone to point me to the correct direction. I am pretty new 
to apache web server administration.


the Set-Cookie header can set multiple cookies.
try adding this line:
Set-Cookie MyTestCookie=MyTestValue;MyOtherTestCookie=MyOtherTestValue
and load the page, then check if the cookies are set.

Under firefox, the cookies are visible from 
tools->options->privacy->show cookies
Search for your cookie name or for the domain that you registered the 
cookies with (defaultly, the domain that sent the headers)


--

Morgan Gangwere
Unknown Software

begin:vcard
fn:Morgan Gangwere
n:Gangwere;Morgan
org:Unknown Software;Programming
adr;dom:;;Contact for details.
email;internet:0.fracta...@gmail.com
title:Developer
tel;work:+1 505 514 2863
note;quoted-printable:Find me on IRC:=0D=0A=
	~indrora!fa...@irc.freenode.net=0d=0a=
	=0D=0A=
	via IM:=0D=0A=
	MSN: fractalsca...@passport.com=0d=0a=
	Yahoo: trogdor_t_b=0D=0A=
	GTalk (talk.google.com SSL): 0.fracta...@gmail.com=0d=0a=
	=0D=0A=
	I'm all over the net!
url:http://sonof.bandit.name/
version:2.1
end:vcard

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

[us...@httpd] 'safe' way dismount a usb drive?

[Foo JH]

Hi all,

I've managed to mount/ umount a usb drive. But I'm not sure if there is
any other commands I need to execute - as a best practice - before I
physically plug it out.

I'm not able to find any documentation online regarding this. But I
thought I check with you guys before I share with my team.

Thanks.

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [us...@httpd] 'safe' way dismount a usb drive?

[Scott Gifford]

Foo JH  writes:

> I've managed to mount/ umount a usb drive. But I'm not sure if there is
> any other commands I need to execute - as a best practice - before I
> physically plug it out.

Not sure what this has to do with Apache; are you serving Web files
from the USB drive?

At any rate, unmounting the drive should do everything necessary to
prepare it for removal.

Scott.

-
The official User-To-User support forum of the Apache HTTP Server Project.
See http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
   "   from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

[us...@httpd] Troubleshooting Apache Out of Memory Errors

[Plutocrat]

I've been struggling with a webserver which crashes every few days with the 
dreaded "out of memory" problem. I've been trying to correlate it to spikes in 
traffic, but that doesn't seem to be the case. I was quite excited last time 
when the crash co-incided with a visit by several search engines, including 
being hammered by the Baiduspider, but the most recent crash was in a 
relatively quiet traffic period, so I'm once again scratching my head. I 
thought, therefore that I'd consult the experts ... 

The thing which I'm most confused about is that the server isn't particularly 
busy. The server seems to die when we get a burst of over 400 page requests in 
an hour, which doesn't seem like it should really be taxing a server of these 
specs:

SPECS
==
First the hardware spec. Its a real (as opposed to virtual) server with 2Gb of 
RAM, and 4Gb swap. It's got an  Intel(R) Pentium(R) Dual  CPU  E2160  @ 
1.80GHz, and a Western Digital 160Gb ATA disk. 

As for software, here's a quick summary. Running on Centos 5.2:
 - Apache/2.2.3 using prefork
 - PHP 5.1.6 (cli) 
 - mysql  Ver 14.12 Distrib 5.0.45
 - Joomla 1.5.7 (latest version)
 - Wordpress 2.x (latest version) 

CONFIG SETTINGS

The relevant sections from my Apache config. 

KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5


StartServers   2
MinSpareServers2
MaxSpareServers4
ServerLimit  256
MaxClients   256
MaxRequestsPerChild  4000


In php.ini I have the following memory-related settings. 
safe_mode = Off
max_execution_time = 30
max_input_time = 60 
memory_limit = 100M 
log_errors = on
report_memleaks = On
error_log = /var/log/php_error.log
post_max_size = 8M

DIAGNOSTICS
===
I wrote a script to capture various bits of memory information about the server 
and set it to go off once every 15 minutes. Of course once the server freezes 
up, the frequency that these cron jobs run at drops, but last crash I managed 
to get one just before and one just after the first Out Of Memory report. Let 
me know if you're interested in a copy of the script. 

There are no relevant error messages in the PHP error log or MySQL error log.

Before crash.
=
=== SUMMARY 
Tue Dec 23 19:15:01 HKT 2008

=== uptime ==
 19:15:01 up 3 days, 11:34,  0 users,  load average: 5.11, 2.31, 1.00

== free -m ==
 total   used   free sharedbuffers cached
Mem:  2001879   1121  0 19351
-/+ buffers/cache:508   1493
Swap: 4094  0   4094

= vmstat 1.5 
procs ---memory-- ---swap-- -io --system-- 
-cpu--
 r  b   swpd   free   buff  cache   si   sobibo   in   cs us sy id wa st
 3  1124 1148504  19920 36043600 930   13   35 10  1 89  0  0
 0  0124 1224564  19924 36051600 0 4 1029  249  8  1 91  1  0
 0  0124 1224572  19924 36051600 0 0 1021  226  0  0 100  0 
 0
 0  0124 1224588  19924 36054800 0 0 1017  229  0  0 100  0 
 0
 1  0124 1286712  19932 36054000 0   236 1023  436  6  0 94  0  0

== ps top 20 Processes by CPU ===
USER %MEM %CPU   PID CMD
apache2.0 18.7 14356 /usr/sbin/httpd
apache2.0 17.3 14357 /usr/sbin/httpd
apache3.2 13.8 14340 /usr/sbin/httpd
apache2.0 13.5 14325 /usr/sbin/httpd
apache3.2 10.8 14330 /usr/sbin/httpd
mysql 2.3  1.1  2569 /usr/libexec/mysqld --basedir=/usr 
--datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid 
--skip-external-locking --socket=/var/lib/mysql/mysql.sock
root  0.5  0.0 12267 /usr/sbin/httpd
root  0.0  0.0  1580 [kjournald]
root  0.0  0.0   227 [kswapd0]
root  0.0  0.0   226 [pdflush]
root  0.0  0.0  2326 pcscd
ntp   0.2  0.0  2477 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g
root  0.7  0.0  2787 /usr/bin/python -tt /usr/sbin/yum-updatesd
root  0.0  0.0 1 init [3]  
680.1  0.0  2767 hald
root  0.0  0.0  2097 auditd
root  0.0  0.0   225 [pdflush]
root  0.0  0.0   422 [kjournald]
root  0.0  0.0   483 /sbin/udevd -d

After crash.
=
=== SUMMARY 
Tue Dec 23 19:46:32 HKT 2008

=== uptime ==
 19:46:34 up 3 days, 12:05,  0 users,  load average: 111.60, 108.31, 85.79

== free -m ==
 total   used   free sharedbuffers cached
Mem:  2001   1987 14  0  1 20
-/+ buffers/cache:   1964 36
Swap: 4094   4067 27

=

Re: [us...@httpd] Unable to compile Apache-2.2.11 on Windows using Cygwin

[sathya sai]

No, I couldn't find .mak/.dep files on
httpd-2.2.11-win32-src.zip
downloaded
from http://httpd.apache.org/download.cgi which retricts me to build through
command line through Cygwin.

I actually have an automated script which compiles the apache source through
nmake (VC command line as mentioned in "Command-Line Build" section on
http://httpd.apache.org/docs/2.2/platform/win_compiling.html URL) from
Cygwin.

This was working fine till httpd-2.2.9 version. But, now its failing now
with http-2.2.11 version.

Could you please let me know if there is any intention behind the removal of
.mak/.dep files from latest version of apache. I couldn't find any reason in
apache portal.

If there is a genuine reason in removing these files, could you please
suggest me an alternate means to compile Apache source through command-line
on Windows. So that I could just tune my existing script to work with the
latest apache source.

Thanks a lot in advance.

On Sun, Jan 4, 2009 at 9:46 PM, Morgan Gangwere <0.fracta...@gmail.com>wrote:

>  William A. Rowe, Jr. wrote:
>
>> sathya sai wrote:
>>
>>>  By looking at the difference in the source tree between Apache-2.2.9 &
>>> Apache-2.2.11, I could see that "*.mak" & ".dep" are removed from the
>>> latest source tree of Apache-2.2.11 & thus causing the problem.
>>>
>>
>> nope - .mak/.dep files are added to the httpd-2.2.x-win32-src.zip package.
>>
>> If you want a native build from the command line (sounds like you do) you
>> just need to grab the right package and ensure ms tools/sdk are in your
>> path.
>>
>> FYI the other way 'round is to run srclib\apr\build\cvtdsp -2005
>> and then load apache.dsw into devenv (visual studio 2003+++).  Even
>> running the command line make when then run them through the console
>> devenv mode from the imported .sln / .vcproj files.
>>
>> If you want a command line cyg/mingw build you don't do any of the above,
>> including using makefile.win.  You run ./configure and proceed as if you
>> were running in any other unix package.
>>
>>
>>
> ...Also, why are you (a) compiling something in an odd way and (b)
> compiling under Cygwin for Windows when you'll get better perfomance on a
> standard win32 build.
>
> --
>
> Morgan Gangwere
> Unknown Software
>
> -
> The official User-To-User support forum of the Apache HTTP Server Project.
> See http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
>   "   from the digest: users-digest-unsubscr...@httpd.apache.org
> For additional commands, e-mail: users-h...@httpd.apache.org
>

Re: [us...@httpd] Troubleshooting Apache Out of Memory Errors

[Sander Temme]

On Jan 4, 2009, at 9:41 PM, Plutocrat wrote:


- Apache/2.2.3 using prefork
- PHP 5.1.6 (cli)


What do you mean when you say 'cli'?  Are you using mod_php, or  
running it as a CGI?



- mysql  Ver 14.12 Distrib 5.0.45
- Joomla 1.5.7 (latest version)
- Wordpress 2.x (latest version)




MaxClients   256



memory_limit = 100M


I've seen Joomla use a lot of memory.  I've used it on a Windows  
server, and have seen the single 100 thread child process balloon to  
800Mb and stay there.  If you get 256 child processes and have each  
allocate 100Mb of memory, you end up with 25Gb memory allocated in  
your 4Gb swapspace: that is just not going to fit.


Note that memory_limit is per script executed.  I believe the default  
is 8Mb: what made you change that?


OK, if you're still with me, thanks for getting this far. So before  
the Out of Memory, the CPU load is around 70% and the load average  
is high, but not critical. After the Out of Memory, the entre Swap  
is full, the load average is insane, and the disk is swapping like  
crazy. There also seem to be a lot of httpd processes spawned, but  
not really doing much. At this point the server is inaccessible.  
Over the next hour or two the swap never really empties, and only  
returns to normal after a reboot.


I'd look at a couple of things:

1) How does your MySQL server configuration match that Apache  
configuration?  Does

   your httpd ever have problems connecting to MySQL?
2) When you look at your processes in top, does the RSS column for  
your httpd
   processes show any disturbing development?  How big do the  
processes get

   in the 4000 connections you allow them?
3) When you filter all the 404 ("File does not exist") out of your  
error log, are

   you left with any clues?

If I were to take a shot at your problem, I'd look in the direction of  
a scenario where your child processes get bigger than 8Mb, and  
something makes child processes hang sufficiently long that the parent  
has to spawn more children to service requests.  Those children then  
also hang on the same resource, which causes more children to be  
spawned to serve incoming requests, eventually filling up your memory  
you'll break down rapidly from there.


S.

--
Sander Temme
scte...@apache.org
PGP FP: 51B4 8727 466A 0BC3 69F4  B7B8 B2BE BC40 1529 24AF





smime.p7s
Description: S/MIME cryptographic signature