[users@httpd] Question on correlation bewteen timeout types (Acquire, connection, proxytimeout and timeout)

[Christian Georg]

Hi all,

I am currently trying to understand the different timout parameters a bit 
better and how they interact.  We are using apache httpd 2.4 mod_proxy on a 
centos system as a reverse proxy with mod_proxy_balancer.

Here is the relevant section from our setup:

Define CONNECTION_OPTIONS "min=10 max=500 smax=30 acquire=5000 
connectiontimeout=5 disablereuse=off keepalive=off timeout=29 ttl=10 retry=60"

BalancerMember https://Server1/   ${CONNECTION_OPTIONS}
BalancerMember https:// Server2/  ${CONNECTION_OPTIONS} 
status=+H
ProxySet lbmethod=bytraffic


What I am trying to understand is how the different timeout parameters interact 
and at what time a request might have been sent to the backend.

Can you confirm/correct the following understanding and answer the questions?

Acquire:
Acquire timeout is a timeout send to the client if the connection pool is very 
busy and apache did not even start to process the request. If this timeout 
happens the request did not reach the proxy and I can be sure no request is 
sent to the backend. In this case, the client will receive an http 503. Should 
I see these kind of requests within the access log of the proxy virtual host?

Connectiontimeout:
Connectiontimeout is a timeout that happens during handshake between the apache 
proxy and the backend. As we are using SSL between the proxy and the backend 
this can happen during SSL Handshake. If this timeout occurs the request was 
trying to be processed by the proxy and will appear in the access log with an 
http code of 504. If this timeout occurs, then I can be sure that the request 
has not been sent to the backend. Can any other http code occur from time 
timeout and is there even a small chance that we might have send the request to 
the backend?

ProxyTimeout and timeout:
Timeout and proxy timeout will both occur only if a successful connection to 
the backend was established and if the request has been sent to the backend. 
Hence I should the see request in both logfiles. If the server does not respond 
within the time defined in this timeout I would expect an http 502 to be 
created. Is this correct? The Part I would like to understand is: does the 
timeout include the acquire time and connection time or is it independent?
Let's take an example: If it takes 4 seconds to acquire a connection and 
another 4 seconds to connect to the backend will the will the client see a 
timout after 29 seconds (timeout includes acquire and connection)  will the 
client see a timeout after 33 seconds (timout does nto include acquire time but 
toes include connection time) or will the client see a timeout after 37 seconds 
(timeout will only start to after the request was send).

Is there any reason to set the proxytimeout to another value than the timeout?

One related question: We are currently logging the http codes using "%>s" from 
my understanding this is the http code we are sending in the response to the 
client. Is this correct and is there a way to also log the http code we have 
received from the backend we are proxying to? I would expect the http code from 
the backend to be empty if a timeout occurs in our systems

Thanks in advance

Chris

[users@httpd] apache 2.2 troubleshooting

[Erik Dobák]

Hi,

i have an apache 2.2 used as a reverse proxy with caching in mem.
Everything is working fine but after a few days the reverse proxy
starts to send all the requests to the backend server.

Is there a way to see what objects are cached in mem? Or should i
switch to disk cache to see that?

Regards

E

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] root perms / rewritemap prg / module system()

[Ben RUBSON]

On 2016-06-08 at 14:24, Nick Kew wrote :
> On Wed, 2016-06-08 at 08:01 -0400, Eric Covener wrote:
>> On Sat, Apr 16, 2016 at 6:00 PM, Ben RUBSON  wrote:
>>> Then my question is, could it be possible ?
>> 
>> You would need your own daemon launched during an early hook (like
>> post_config).  You wouldn't be able to respond [directly] to requests,
>> you'd need to reach out over something like a pipe the way rewritemap
>> does.
>> 
> A hacked suexec would be a per-request option.  Or using
> solaris and mod_privileges might just possibly help with
> more fine-grained escalation.

Nick, thank you very much for your answer, and sorry for my late reply.

I red suexec source code, it gave me some ideas :
I could write my own setuid-ed program which would :
- setuid()
- only perform the needed tasks under 
- exit
This program could be then called from any other custom module etc...

Thank you again !

Ben


-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] root perms / rewritemap prg / module system()

[Stormy]

At 03:53 PM 6/24/2016 +0200, Ben RUBSON wrote:

On 2016-06-08 at 14:24, Nick Kew wrote :
> On Wed, 2016-06-08 at 08:01 -0400, Eric Covener wrote:
>> On Sat, Apr 16, 2016 at 6:00 PM, Ben RUBSON  wrote:
>>> Then my question is, could it be possible ?
>>
>> You would need your own daemon launched during an early hook (like
>> post_config).  You wouldn't be able to respond [directly] to requests,
>> you'd need to reach out over something like a pipe the way rewritemap
>> does.
>>
> A hacked suexec would be a per-request option.  Or using
> solaris and mod_privileges might just possibly help with
> more fine-grained escalation.

Nick, thank you very much for your answer, and sorry for my late reply.

I red suexec source code, it gave me some ideas :
I could write my own setuid-ed program which would :
- setuid()
- only perform the needed tasks under 
- exit
This program could be then called from any other custom module etc...


Maybe esoteric, possibly off-topic for this precise thread, but would 
mod-itk be potentially useful? I looked into it a few years back, it 
*seemed* to do what was promised, but also carried a fairly major cpu-cycle 
overhead. I didn't have time to try and refine implementation and never put 
it into production...


Best -- Paul 



-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] root perms / rewritemap prg / module system()

[Ben RUBSON]

> On 24 Jun 2016, at 16:24, Stormy  wrote:
> 
> At 03:53 PM 6/24/2016 +0200, Ben RUBSON wrote:
>> On 2016-06-08 at 14:24, Nick Kew wrote :
>> > On Wed, 2016-06-08 at 08:01 -0400, Eric Covener wrote:
>> >> On Sat, Apr 16, 2016 at 6:00 PM, Ben RUBSON  wrote:
>> >>> Then my question is, could it be possible ?
>> >>
>> >> You would need your own daemon launched during an early hook (like
>> >> post_config).  You wouldn't be able to respond [directly] to requests,
>> >> you'd need to reach out over something like a pipe the way rewritemap
>> >> does.
>> >>
>> > A hacked suexec would be a per-request option.  Or using
>> > solaris and mod_privileges might just possibly help with
>> > more fine-grained escalation.
>> 
>> Nick, thank you very much for your answer, and sorry for my late reply.
>> 
>> I red suexec source code, it gave me some ideas :
>> I could write my own setuid-ed program which would :
>> - setuid()
>> - only perform the needed tasks under 
>> - exit
>> This program could be then called from any other custom module etc...
> 
> Maybe esoteric, possibly off-topic for this precise thread, but would mod-itk 
> be potentially useful? I looked into it a few years back, it *seemed* to do 
> what was promised, but also carried a fairly major cpu-cycle overhead. I 
> didn't have time to try and refine implementation and never put it into 
> production...

Thank you for your suggestion Paul !
I used to use mpm-itk, but I reverted back to built-in MPMs as I'm not really 
confident having the main server (the very first one receiving requests) 
running as root.

Best regards,

Ben


-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] Last-Modified header overridden

[Vacelet, Manuel]

On Sun, Jun 19, 2016 at 3:17 PM, Luca Toscano 
wrote:

>
>
> 2016-06-08 16:14 GMT+02:00 Vacelet, Manuel :
>
>> On Tue, Jun 7, 2016 at 11:02 PM, Luca Toscano 
>> wrote:
>>
>>>
>>>
>>> 2016-06-07 10:55 GMT+02:00 Vacelet, Manuel :
>>>


 On Mon, Jun 6, 2016 at 5:32 PM, Vacelet, Manuel <
 manuel.vace...@enalean.com> wrote:

> dOn Mon, Jun 6, 2016 at 5:00 PM, Vacelet, Manuel <
> manuel.vace...@enalean.com> wrote:
>
>> On Mon, Jun 6, 2016 at 4:09 PM, Luca Toscano 
>> wrote:
>>
>>>
>>> I was able to repro building httpd from 2.4.x branch and following
>>> your configuration files on github. I am almost sure that somewhere 
>>> httpd
>>> sets the Last-Modified header translating "foo" to the first Jan 1970 
>>> date.
>>> I realized though that I didn't recall the real issue, since passing 
>>> value
>>> not following the RFC can lead to inconsistencies, so I went back and
>>> checked the correspondence. Quoting:
>>>
>>> "Actually I wrote this snippet to highlight the behaviour (the
>>> original code sent the date in iso8601 instead of rfc1123) because it 
>>> was
>>> more obvious.
>>> During my tests (this is extracted from an automated test suite),
>>> even after having converted dates to rfc1123, I continued to get some
>>> sparse errors. What I got is that the value I sent was sometimes 
>>> slightly
>>> modified (a second or 2) depending on the machine load."
>>>
>>> So my understanding is that you would like to know why a
>>> Last-Modified header with a legitimate date/time set by a PHP app gets
>>> "delayed" by a couple of seconds from httpd, right?
>>>
>>
>> Yes for sure, this is the primary issue.
>> However, the (undocumented) difference of behavior from one version
>> to another (2.2 -> 2.4 and more surprisingly from between two 2.4 
>> versions)
>> is also in question here.
>> Even more strange, 2.4 built for other distrib doesn't highlight the
>> behaviour !
>>
>>
>
> I made another series of test and it seems to be linked to fastcgi.
>
> I took the stock apache (2.4.6 plus tons of patches)  & php-fpm
> (5.4.16 + tons of patches) from RHEL7 and I get the exact same behaviour
> (headers rewritten to EPOCH)
> However, if I server the very same php script from mod_php (instead of
> fcgi) it "works" (the headers are not modified).
>
>
 For the record, I also have the same behaviour (headers rewritten when
 using php-fpm + fastcgi) on alpine linux 3.4 that ships apache2-2.4.20.
 So AFAICT, it doesn't seem distro specific.

 On the root of the problem, from my point of view:
 - the difference between mod_php vs. php-fpm + fcgi is understandable
 (even if not desired and not documented).
 - the fact that fcgi handler parse & rewrite headers seems to lead to
 inconsistencies (I'll try to build a test case for that).
 - however, even if the headers are wrong, I think apache default (use
 EPOCH) is wrong as it leads to very inconsistent behaviour (the resource
 will never expire). I would prefer either:
 -- do not touch the header
 -- raise a warning and discard the header

 What do you think ?

>>>
>>>
>>> From my tests the following snippet of code should be responsible for
>>> the switch from 'foo' to unix epoch:
>>>
>>> *https://github.com/apache/httpd/blob/2.4.x/server/util_script.c#L663
>>> *
>>>
>>> The function that contains the code, ap_scan_script_header_err_core_ex,
>>> is wrapped by a lot of other functions eventually called by modules like
>>> mod-proxy-fcgi. A more verbose description of the function in:
>>>
>>> https://github.com/apache/httpd/blob/2.4.x/include/util_script.h#L200
>>>
>>> Not sure what would be the best thing to do, but probably we could
>>> follow up in a official apache bugzilla task?
>>> https://bz.apache.org/bugzilla/enter_bug.cgi?product=Apache%20httpd-2
>>>
>>>
>> Wow, thanks for the investigation !
>>
>
> Sorry for the delay! I submitted a patch for trunk with a possible fix,
> namely dropping (and logging at trace1 level) any non compliant date/time
> set in a Last-Modified header returned by a FCGI/CGI script:
> http://svn.apache.org/r1748379
>
>
Cool :)


> The fix is also in the list of proposal for backport to the 2.4.x branch,
> we'll see what other people think about this solution.
>
> We should also do a follow up for the other main issue, namely the fact
> that you see a different/delayed Last-Modified header sometimes among your
> FCGI/httpd responses. Can you give me an example of Last-Modified header
> value before/after the "delay" and a way to repro it?
>

I wrote a test case in the "time" branch:
https://github.com/vaceletm/bug-httpd24/tree/time

In my own tests, I get:
->8

RE: [users@httpd] Issues migrating Weblogic proxies from Sun One 6.1 to Apache 2.4

[Joe Muller]

 Just an update on this issue, if anybody is following. We have eliminated the 
Load Balancer and the Firewall as a possible cause.  Our last test was directly 
between the IE client browser and the Apache 2.4 proxy server, resulting in the 
same errors in the Apache logs similar to this:

>>*Exception type] [WRITE_ERROR_TO_CLIENT raised at line 560 of  
>> BaseProxy.cpp, refer:  
>> https://intgalf.xyz.com/ALFA/selectFileType.do?fileType=O1MM

  We also tested using different browsers (IE, Fire Fox, Chrome, and had mixed 
results, but eventually we saw the same errors from the Weblogic 12c plug-in 
the Apache logs at some point.

 So at this point it is down to the Apache proxy and/or the Oracle Weblogic 12c 
Plug-in for Apache.  Anybody have any ideas ?  I am out of them right now :-(

Perhaps it has something to do with  the Apache 2.4 compilation options ?  If I 
had known  before that going with Apache 2.4 instead of IPlanet would have been 
this difficult we would have just purchased Oracle HTTP Server.  At least we 
would have full Oracle support through and through, although that may not be 
saying much, since it is based on Apache 2.2. Also I am worried about 
Oracle compatibility with 3rd party plug-ins like RSA Access Manager (SSO) and 
Tea Leaf.  

To be honest I am surprised Oracle doesn't give more guidance on this.   I 
would think we wouldn't be the first people to encounter this, but based on the 
lack of information in their knowledge base and the response from their 
technical support it seems otherwise.

Thanks,

-Joe


-Original Message-
From: Mike Rumph [mailto:mike.ru...@oracle.com] 
Sent: Monday, June 20, 2016 4:08 PM
To: users@httpd.apache.org
Subject: Re: [users@httpd] Issues migrating Weblogic proxies from Sun One 6.1 
to Apache 2.4

Hello Joe,

It seems to me that mod_reqtimeout applies to the amount of time a allowed for 
the server to receive the request from the client  (Read from client).

But your errors are indicating a problem on the Write to client.
As I am understanding our WLS plugin developer, the problem seems to be that 
when the server has readied a response to send back to the client, the client 
has already dropped the connection.
So in your topology this could be client browser, firewall or load balancer.

Have you considered the email responses from Todd Simons and Daniel?
The response from Daniel concerning the load balancer may be the most pertinent.

Thanks,

Mike

On 6/20/2016 11:11 AM, Joe Muller wrote:
>   I checked our httpd.conf and we do have reqtimeout_module already loaded, 
> however there is nothing set.  According to the Apache 2.4 documentation the 
> default values are:
>
> Default: header=20-40,MinRate=500 body=20,MinRate=500
>
> Wouldn't you think that should be high enough ?  I know for a fact 
> that the data is not taking 20-40 seconds to post in these 
> applications, as the users are not waiting that long.  They are only 
> seeing slowdowns of a few seconds between pages (which in network time 
> is pretty long, ecspecially accessing the applications locally and not 
> coming in from over the internet.)
>
>
> -Original Message-
> From: Mike Rumph [mailto:mike.ru...@oracle.com]
> Sent: Monday, June 20, 2016 10:45 AM
> To: users@httpd.apache.org
> Subject: Re: [users@httpd] Issues migrating Weblogic proxies from Sun 
> One 6.1 to Apache 2.4
>
> Hello Joe,
>
> I am not in a position to offer an official Oracle statement for your 
> situation.
> But I do work closely with the owners of the WLS plug-ins.
>
> Here is an initial evaluation from one of the developers:
>
> "WRITE_ERROR_TO_CLIENT is typically seen when there is an error 
> writing the response to the client (WLS plug-in client). This usually 
> occurs when the user sends a request, but closes the browser (or hits 
> a stop
> button) before the response is received by the client. In such a scenario, 
> from the plug-in perspective, whenever response is received from WLS, it 
> cannot relay it to the client as the connection is broken, and it logs 
> WRITE_ERROR_TO_CLIENT error. This is usually a harmless error. If the above 
> is not true (closing the browser etc), then it may be possible that the 
> client timeout is too low (or lower than the WLS response time for the 
> request). In such cases, the timeout needs to be increased. I am not aware of 
> what parameters to look out for here, but mod_reqtimeout may be a good 
> beginning."
>
> I hope that this is helpful to you.
> For more specific details, you would still need to contact official Oracle 
> support channels.
>
> Thanks,
>
> Mike
>
> On 6/17/2016 7:30 AM, Joe Muller wrote:
>> I am working on a project to migrate all our IPlanet 6.1 SP19 webserver 
>> proxies (formerly Sun One) to Apache 2.4, since IPlanet 6.1 does not support 
>> TLS 1.2 and IPlanet 7.0 is being EOL. Our backend application servers are 
>> Weblogic 9.2 / Weblogic 12c. The IPlanet proxies have performed FLAWLESSLY 
>> for ove