SSLProxyEngine On Was already turned on this entire time inside the ssl.conf (I include it) VirtualHost section.
I am debating turning on HTTPS in Tomcat on the backend 10.x.x.x app server, and then HTTPS the whole way through and see if that makes any difference. I may need to request a new firewall to be opened, which is not able to open until 3/2. Gonna see. On Thu, Feb 23, 2017 at 8:22 AM, Antonio S. Cofiño <cofi...@gmail.com> wrote: > > On 23/02/17 12:43, André Warnier (tomcat) wrote: > >> On 22.02.2017 19:22, Aaron Gray wrote: >> >>> So this is interesting. >>> >>> So from HTTP server #1 (172.1.1.1 example) I hit: >>> https://172.1.1.1:23270/static and I see this in the HTTP log: >>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 32 >>> I see this in the Tomcat log: >>> 172.1.1.1 - - [22/Feb/2017:10:14:48 -0800] "GET /static/ HTTP/1.1" 200 32 >>> ((( Yes I know they look identical, but the logformat is the same, so it >>> comes out looking the same, but they are two diff log files totally ))) >>> >>> I then hit the https://loadbalancer.domain.com/static from my Win10 >>> laptop >>> (10.1.1.2 example) >>> and I see this in the HTTP server #1 log, but NOTHING in the tomcat >>> access >>> log. >>> 10.1.1.2 - - [22/Feb/2017:10:20:02 -0800] "-" 408 - >>> >>> So 408. Timeout. Hmm... Why? We KNOW that it can connect from http -> >>> tomcat:18080 perfectly. So is it a timeout BACK to the F5? That i dont >>> know yet. >>> >> >> It looks as if, in that case, httpd is trying to connect to tomcat, but >> - either it /is/ connecting, and sending the request, but tomcat never >> answers >> - or it is connecting to something else, which isn't tomcat >> (but which accepts the request and doesn't answer either) >> >> I am not *sure*, but I believe, that if httpd was trying to connect to >> something that isn't tomcat, and is not listening on port 18080, it would >> get a different error, and a different logfile message. So it would seem >> that at least the TCP connection works. >> > > > Is the F5 loadbalancer configured to make reverse proxy to a SSL http? > The directive SSLProxyEngine onthe F5 loadbalencer has to be enabled. > http://httpd.apache.org/docs/current/mod/mod_ssl.html#sslproxyengine > > If the loadbalancer it's delegating the SSL work to the server1 > server1 https listen = 172.1.1.1:23270 > > then the F5 could be connecting as http regular TCP connection to a SSL > TCP one and therefor the SSL handshake it's not happening. > > Antonio > > > > > > >> Now there is still a puzzle there : when you connect as >> browser --> httpd --> tomcat >> then everything works : httpd connects to tomcat, sends a request, gets a >> response, and returns it to the browser. >> >> But when you do the same as : >> browser --> F5 --> httpd --> tomcat >> it "does not work", and the problem seems to be at httpd --> tomat >> Why would it make a difference in httpd --> tomcat, if it is either the >> browser or the F5 which sends the request to httpd ? >> the httpd --> tomcat connection should be the same, no matter what. >> >> Hmm, for now as puzzled as you are. >> Maybe we have a quantum phenomenon here, where the mere fact of you >> looking at the logfile, changes what happened beforehand. Just kidding. >> >> Informational note : the Apache access log line, is written when Apache >> *has finished* processing the request (including the possible round-trip >> through tomcat) and has sent the response to the client. (That's because, >> for example, it can also write the time spent, and the size of the >> response; so it has to know those before it can write the log line). >> I presume that it is the same for tomcat. >> Just mentioning this, so that you would maybe have a second look at the >> tomcat access log, after a few minutes, to make sure that there really is >> no corresponding line there. >> It could be that tomcat actually /is/ receiving the request, and >> processing it, but that it takes a long time, and that httpd tires of >> waiting and already times out in the meantime. In such a case, you would >> probably see an error some time later in the tomcat error log, when it is >> trying to eventually return the response, and finds a closed connection >> instead. >> Now again, why it would do that in one case and not the other, is still a >> mystery. >> >> Another note : in the "timeout" log line above, httpd is not showing the >> request URL which triggered this timeout. >> a) it may be showing some additional information in the Apache error log >> b) these exists an httpd add-on module (mod_log_forensic ?) which could >> provide more details in the log, about what happens precisely >> >> >> >>> On Tue, Feb 21, 2017 at 3:39 PM, André Warnier (tomcat) <a...@ice-sa.com> >>> wrote: >>> >>> On 21.02.2017 23:28, Aaron Gray wrote: >>>> >>>> Antonio: The Tomcat server has no knowledge of the F5, or that it is >>>>> being >>>>> fronted by an Apache HTTP Server. I do SSL termination in Apache HTTP >>>>> Server, and clear-text from HTTP to Tomcat. >>>>> My redirect port for the normal HTTP listen in Tomcat is commented out. >>>>> <Connector port="18080" protocol="HTTP/1.1" >>>>> connectionTimeout="20000" /> >>>>> <!-- A "Connector" using the shared thread pool--> >>>>> <!-- >>>>> <Connector executor="tomcatThreadPool" >>>>> port="8080" protocol="HTTP/1.1" >>>>> connectionTimeout="20000" >>>>> redirectPort="8443" /> >>>>> --> >>>>> >>>>> Andre: >>>>> The URL I am using is https://loadbalancer.domain.com >>>>> It is listening on port 80 and 443, if you hit 80, internally it >>>>> redirects >>>>> you to 443. No SSL cert on the F5 load balancer. It simply sends the >>>>> traffic to one of the two HTTP servers (round-robin, also tried >>>>> persistence, no difference). The HTTP server is listening only for >>>>> HTTPS >>>>> on 23270/tcp. >>>>> When I access these DMZ webservers via the F5 load balancer (to which I >>>>> dont have access to, but the network folks configure for me), it hangs. >>>>> Eventually returns: server1 https listen = 172.1.1.1:23270 >>>>> >>>>> https://loadbalancer.domain.com:23270/SelfService >>>>> cant load. >>>>> >>>>> No idea why the URL is being re-written with the ":23270". >>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test: >>>>> <Context docBase="/path/to/tomcat/static" path="/static" /> >>>>> Then put a simple index.html in there. Accessing via the Apache Web >>>>> Servers works fine, but if you hit it with the Load Balancer it once >>>>> again >>>>> adds the https://loadbalancer.domain.com:23270/static >>>>> >>>>> Hitting https://loadbalancer.domain.com >>>>> I see my "Hello world!" which is all that is in index.html. This is the >>>>> DocumentRoot of HTTP, and *not* proxied over at this time. >>>>> >>>>> >>>> So in this case, there is no delay, and you get the Apache httpd-hosted >>>> "index.html" containing "Hello World. Right ? >>>> >>>> Only >>>> >>>> /SelfService and /static are proxied >>>>> /static just being my test of static content, but still served up by >>>>> Tomcat.. >>>>> >>>>> It's exactly 30 seconds before the page cannot be loaded when trying >>>>> anything proxied to Tomcat, but also accessed via the F5 load balancer. >>>>> Not sure where the 30 seconds comes from; perhaps a load balancer time >>>>> out, >>>>> as I dont see a "30" in my httpd configurations or my tomcat server.xml >>>>> >>>>> >>>>> You can certainly look at the Apache httpd logs, and the tomcat logs, >>>> to >>>> see if you get a request or not. >>>> In Apache httpd, you can set the loglevel individually for mod_proxy (if >>>> you are running v 2.4), and it should show something if it gets this >>>> request and forwards it to tomcat. >>>> In tomcat, you can either enable an access log (which will show if it >>>> receives this request), or you could temporarily remove/rename the >>>> /static >>>> webapp. This way, it should trigger an error "not found" which you would >>>> also see in the error log. >>>> >>>> There should be nothing between them to hinder it. We have many load >>>> >>>>> balancers and this one specifically you dont need to open any firewall >>>>> requests for the specific networks the HTTP servers are on. I did have >>>>> to >>>>> get the firewall opened up to allow me to hit >>>>> https://loadbalancer.domain.com because the VIP for " >>>>> loadbalancer.domain.com" >>>>> is in the DMZ, and my Desktop & VPN networks cannot hit it on 80/443 >>>>> without opening holes. But beyond that, any connection from the F5 to >>>>> the >>>>> HTTP Server should be 100% open bi-directional, since same subnet. >>>>> >>>>> >>>>> But something isn't working, otherwise you would not be asking. >>>> So, >>>> >>>> a) hitting the tomcat webapps through httpd seems to be working fine >>>> (browser -> httpd:23270 -> tomcat:18080 -> webapp or static) >>>> >>>> b) hitting a non-proxied-to-tomcat resource of httpd seems to work fine >>>> too, even through the F5 >>>> (browser -> F5:443 -> httpd:23270 -> html page) >>>> >>>> c) it is only when you do : >>>> (browser -> F5:443 -> httpd:23270 -> tomcat:18080 -> webapp or >>>> static) >>>> that you see this issue >>>> >>>> It would really help if you looked in the logs of both httpd and tomcat, >>>> and checked for differences betweens cases a, b and c above. >>>> >>>> I believe that the F5 message with the port 23270 is a minor issue, of >>>> information disclosure by the F5, that it should not disclose. >>>> >>>> But the reason why it returns this error is obviously that in that case, >>>> it does not get a response from his request to httpd. >>>> The reason for this response not coming back to the F5 (in case c only), >>>> can be due to either httpd or tomcat. But F5 doesn't know about tomcat. >>>> So >>>> for the F5, it is httpd which is not responding. Thus, >>>> - either httpd is never getting the request from the F5 (unlikely, >>>> because >>>> in b above it gets it and responds) >>>> - or httpd is getting the request from the F5, but not forwarding it to >>>> tomcat, but also not returning an immediate error response to F5 (which >>>> seems also unlikely, because of a and b) >>>> - or httpd is getting the request, forwarding it to tomcat, but not >>>> getting a response from tomcat. So >>>> - either tomcat is never getting the request from httpd (but in a, it >>>> gets it) >>>> - or tomcat is getting the request from httpd, but not responding >>>> (but >>>> in a, it does) >>>> - or tomcat is getting the request and responding, but the response >>>> never gets back to httpd (but in a, it does) >>>> >>>> So if a and b and c are all accurate, there is something apparently >>>> illogical happening. >>>> This would lead to the conclusion that a and b and c cannot all be >>>> accurate. >>>> >>>> The logs.. ? >>>> >>>> >>>> >>>> On Tue, Feb 21, 2017 at 2:05 PM, Antonio S. Cofino <cofi...@gmail.com> >>>> >>>>> wrote: >>>>> >>>>> Aaron, on tomcat instances change the redirectPort attributte on the >>>>> http >>>>> >>>>>> conectó to the loabbalancer's port 443 >>>>>> >>>>>> My guess is that your webapp has restriction rule requesting SSL con >>>>>> fidntial channel. Therefore the non-confidential to the 18080 port >>>>>> from >>>>>> the >>>>>> balancer are redirected to the 23270 port, but it should be 443. >>>>>> >>>>>> Antonio >>>>>> >>>>>> >>>>>> >>>>>> El 21/2/2017 19:46, "Aaron Gray" <aaronmg...@gmail.com> escribió: >>>>>> >>>>>> I have an application server from a vendor that comes bundled with an >>>>>> additional Apache Tomcat server. The webapp SelfService.war is vendor >>>>>> supplied too. >>>>>> >>>>>> Here's my problem (IP's replaced to protect the innocent): >>>>>> >>>>>> networks: >>>>>> DMZ=172.x.x.x >>>>>> INTERNAL=10.x.x.x >>>>>> >>>>>> server1 https listen = 172.1.1.1:23270 >>>>>> server2 https listen = 172.1.1.2:23270 >>>>>> F5 load balancer hostname = loadbalancer.domain.com:443 >>>>>> backend tomcat server = 10.1.1.1:18080 >>>>>> >>>>>> mod_proxy configuration: >>>>>> ProxyPass /SelfService http://10.1.1.1:18080/SelfService >>>>>> ProxyPassReverse /SelfService http://10.1.1.1:18080/SelfService >>>>>> >>>>>> When I access these DMZ webservers which mod_proxy back to Apache >>>>>> Tomcat >>>>>> as: >>>>>> https://172.1.1.1:23270/SelfService >>>>>> and >>>>>> https://172.1.1.2:23270/SelfService <https://172.1.1.1:23270/SelfS >>>>>> ervice >>>>>> >>>>>>> >>>>>>> They load properly. Perfectly, every time! >>>>>> >>>>>> When I access these DMZ webservers via the F5 load balancer (to which >>>>>> I >>>>>> dont have access to, but the network folks configure for me), it >>>>>> hangs. >>>>>> Eventually returns: >>>>>> https://loadbalancer.domain.com:23270/SelfService >>>>>> cant load. >>>>>> >>>>>> No idea why the URL is being re-written with the ":23270". >>>>>> I added static content to the server.xml on 10.1.1.1 (Tomcat) to test: >>>>>> <Context docBase="/path/to/tomcat/static" path="/static" /> >>>>>> Then put a simple index.html in there. Accessing via the Apache Web >>>>>> Servers works fine, but if you hit it with the Load Balancer it once >>>>>> again >>>>>> adds the https://loadbalancer.domain.com:23270/static >>>>>> >>>>>> Do you have any thoughts? Thanks so much, I have been working with >>>>>> this >>>>>> for weeks now with no success >>>>>> >>>>>> >>>>>> >>>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>>> For additional commands, e-mail: users-h...@tomcat.apache.org >>>> >>>> >>>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
