Hi Imam. It would be helpfull to know your used versions:
haproxy -vv apache httpd version shibboleth version A small workflow picture like: Client -> haproxy -> apache httpd -> shibboleth ? Am 27.10.2018 um 07:44 schrieb Imam Toufique: > Hi Igor, > > Thanks very much for offering to help! I will do this in sections, > hopefully, I > can keep this from being too cluttered. > > haproxy.cfg: > -------------------------------------------------------------------------------------- > global > #log /dev/log local0 debug > #log /dev/log local1 debug > log 127.0.0.1 local2 > chroot /var/lib/haproxy > stats timeout 30s > user haproxy > group haproxy > tune.ssl.default-dh-param 2048 > daemon > > defaults > log global > mode http > option tcplog > option dontlognull > timeout connect 5000 > timeout client 50000 > timeout server 50000 > timeout tunnel 9h > option tcp-check > > frontend http_front > bind :80 > bind 0.0.0.0:443 ssl crt /etc/haproxy/crsplab2_1.pem > stats uri /haproxy?stats > default_backend web1_cluster > option httplog > log global > #option dontlognull > log /dev/log local0 debug The 2 log entries are redundant, imho. > mode http Is set at default block. Please take a look at https://www.haproxy.com/blog/the-four-essential-sections-of-an-haproxy-configuration/ > option forwardfor # forward IP > http-request set-header X-Forwarded-Port %[dst_port] > http-request add-header X-Forwarded-Proto https if { ssl_fc } I personally would use here also set-header instead of add. > redirect scheme https if !{ ssl_fc } > > acl host_web2 hdr(host) -i crsplab2.oit.uci.edu/webdav This should be only the host name. https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.6-hdr and https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.6-req.hdr I would add a addition acl. acl host_web2 hdr(host) -i crsplab2.oit.uci.edu acl path_web2 path_beg -i /webdav > use_backend webdav_cluster if host_web2 an add it to the use_backend line use_backend webdav_cluster if host_web2 path_web2 > acl host_web3 path_beg /jhub > use_backend web3_cluster if host_web3 > > > backend webdav_cluster > balance roundrobin I would add here the following line. cookie SRV1 insert indirect nocache > server web1 10.1.100.156:8080 check inter 2000 cookie w1 > server web2 10.1.100.160:8080 check inter 2000 cookie w2 > > backend web3_cluster I would add here the following line. cookie SRV2 insert indirect nocache > server publicIP:443 check ssl verify none inter 2000 cookie w1 > ----------------------------------------------------------------------------------------------------- > Note: I have a single backend node, as it was easy to test with just one node, > instead of making changes to 2 nodes at a time. > > Here is my apache config: > > in httpd.conf, only change I have made is ( the rest is a stock centos 7.5 > httpd.conf ): > ------------------------------------- > ServerName 10.1.100.160:80 ( Internal IP of the backend node) > Redirect permanent /jhub https://crsplabweb1.domain.com/jhub > ------------------------------------- > > in my ssl.conf, where I access the jupyterhub instance running in > 127.0.0.1:8000 > <http://127.0.0.1:8000> . Also, note that the backend is running shibboleth > SP. One of the issues I encountered is, If I did not have SSL , i was > getting a > browser warning for not having SSL. Can you set up shibboleth in that manner that he answers with proxy.domain.com? As we don't know which version is in place I send you just as a hint here some links. https://wiki.shibboleth.net/confluence/display/SHIB2/SPReverseProxy http://shibboleth.1660669.n2.nabble.com/shibboleth-sp-behind-an-HAproxy-td5960149.html https://duckduckgo.com/?q=shibboleth+behind+haproxy > Here is my ssl.conf: > > -------------------------------------------------------------------------- > Listen 443 https > SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog > SSLSessionCache shmcb:/run/httpd/sslcache(512000) > SSLSessionCacheTimeout 300 > SSLRandomSeed startup file:/dev/urandom 256 > SSLRandomSeed connect builtin > SSLCryptoDevice builtin > > <VirtualHost _default_:443> > > UseCanonicalName on > ServerName crsplabweb1.domain.com:443 Maybe you can change this to proxy.domain.com and add a ServerAlias ServerName proxy.domain.com ServerAlias crsplabweb1 crsplabweb1.domain.com https://httpd.apache.org/docs/2.4/mod/core.html#serveralias > ErrorLog logs/ssl_error_log > TransferLog logs/ssl_access_log > LogLevel warn > > SSLEngine on > > SSLProtocol all -SSLv2 -SSLv3 > SSLCipherSuite HIGH:3DES:!aNULL:!MD5:!SEED:!IDEA > SSLCertificateFile /etc/pki/tls/certs/crsplabweb1.domain.com_cert.cer > SSLCertificateKeyFile /etc/pki/tls/certs/crsplabweb2.key > SSLCertificateChainFile > /etc/pki/tls/certs/crsplabweb1.domain.com_interm_reverse.c > > <Files ~ "\.(cgi|shtml|phtml|php3?)$"> > SSLOptions +StdEnvVars > </Files> > <Directory "/var/www/cgi-bin"> > SSLOptions +StdEnvVars > </Directory> > > <Location /jhub> > ProxyPass http://127.0.0.1:8000/jhub > ProxyPassReverse http://127.0.0.1:8000/jhub > RequestHeader unset Accept-Encoding > ProxyPreserveHost on > AuthType shibboleth > ShibRequestSetting requireSession 1 > Require shibboleth > ShibUseHeaders On > ShibBasicHijack On > RewriteEngine On > RequestHeader set X-Remote-User %{REMOTE_USER}s > </Location> > > <LocationMatch > "/jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)"> > ProxyPassMatch ws://127.0.0.1:8000/jhub/$1/$2$3 > ProxyPassReverse ws://127.0.0.1:8000/jhub/$1/$2$3 > </LocationMatch> > > BrowserMatch "MSIE [2-5]" \ > nokeepalive ssl-unclean-shutdown \ > downgrade-1.0 force-response-1.0 > > CustomLog logs/ssl_request_log \ > "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b" > </VirtualHost> > ---------------------------------------------------------------------------------- > > Thanks Regards Aleks > On Fri, Oct 26, 2018 at 8:34 PM Igor Cicimov <[email protected] > <mailto:[email protected]>> wrote: > > Hi Imam, > > On Sat, Oct 27, 2018 at 9:37 AM Imam Toufique <[email protected] > <mailto:[email protected]>> wrote: > > Hi, > > I came up with the following config, things seem to be working now, > for > the most part. > > frontend http_front > bind :80 > bind 0.0.0.0:443 ssl crt /etc/haproxy/crsplab2_1.pem > stats uri /haproxy?stats > default_backend web1_cluster > option httplog > log global > #option dontlognull > log /dev/log local0 debug > mode http > option forwardfor # forward IP > http-request set-header X-Forwarded-Port %[dst_port] > http-request add-header X-Forwarded-Proto https if { ssl_fc } > redirect scheme https if !{ ssl_fc } > acl host_web3 path_beg /jhub > use_backend web3_cluster if host_web3 > > web3_cluster > > backend web3_cluster > mode http > balance source > server crsplabweb1.domain.com publicIP:443 check ssl verify none > inter 2000 cookie w1 > > The above config gets me to the backend node -- where I have a > jupyterhub instance running + . Shibboleth SP running for > authentication. As I could not get shibboleth SP to work by staying > in > my private network, I had to set up a public IP for the backend node, > get SSL certs - so shibboleth authentication could be done. I am sure > there is a better approach to this, but I don't know what it is. I > will > be trying out SNAT to see if that will allow me to keep using my > private > IP for the backend nodes. If any of you know how to do SNAT, please > chime in, it would be worth the time/effort to try it out. > > Now, the interesting thing I have noticed with the above setup -- > when I > connect to HAProxy, let's say with https://proxy.domain.com , I > authenticate with shibboleth, and then the URL in the browser points > to > the backend node. > > For example: > > my proxy address: https://proxy.domain.com/jhub > > after I connect to the backend, the URL turns into - > https://crsplabweb1.domain.com/jhub/tree? > > ...and everything works thereafter. > > I tried the rewrite method that Igor has suggested before, that did > not > make any difference. But what I noticed is, after I connect, no > traffic > go through the proxy anymore, my client ( i.e. laptop) connects > directly > to the backend server. Not sure if this good or bad though (?) , but, > I > am not sure how to configure this so that I will go through a proxy > but > still be connected in the backend via a private IP and I can ( still ) > authenticate via shibboleth. > > So, when I change the 'web3_cluster' backend to : > > server crsplabweb1 privateIP:80 inter 2000 cookie w1 > > and, I set backend apache to accept connection on port 80, then I > break > shibboleth authentication. > > Any inputs here? > > thanks, guys! > > > I think it is time for you to provide the full HAP and Apache configs so > we > can see what is going on (please obfuscate any sensitive data). Also the > use > of the "cookie w1" is not clear since you are not setting it in HAP and is > kinda redundant for single backend setup. > > > > On Thu, Oct 25, 2018 at 1:21 AM Igor Cicimov > <[email protected] > <mailto:[email protected]>> > wrote: > > > > On Thu, Oct 25, 2018 at 6:31 PM Igor Cicimov > <[email protected] > <mailto:[email protected]>> wrote: > > > > On Thu, 25 Oct 2018 6:13 pm Imam Toufique <[email protected] > <mailto:[email protected]>> wrote: > > so I almost got this to work, based on the situation I am > in. To elaborate just a bit, my setup involves a > shibboleth > SP that I need to authenticate my application. Since I > can't set up the HA proxy node with shibboleth SP - I had > to > wrap my application in the backend with apache so I can > pass > REMOTE_USER to the application. the application I have > is - > jupyterhub and it start with its own proxy. Long story > short, here is my current setup: > > frontend > bind :80 > bind :443 ssl crt /etc/haproxy/crsplab2_1.pem > stats uri /haproxy?stats > default_backend web1_cluster > option httplog > log global > #option dontlognull > log /dev/log local0 debug > mode http > option forwardfor # forward IP > http-request set-header X-Forwarded-Port %[dst_port] > http-request add-header X-Forwarded-Proto https if { > ssl_fc } > redirect scheme https if !{ ssl_fc } > > acl host_web3 path_beg /jhub > use_backend web3_cluster if host_web3 > > backend > server web1.oit.uci.edu <http://web1.oit.uci.edu> > 128.110.80.5:80 <http://128.110.80.5:80> check > > this works for the most part. But I am confused with a > problem. when I get to my application, my backend IP > address > shows up in the browser URL. > > for example, I see this in my browser: > > http://128.110.80.5/jhub/user/itoufiqu/tree? > > whereas, I was expecting that it would show the original > URL, such as: > > http://crsplab2.domain.com/jhub/user/itoufiqu/tree > <http://crsplab2.domain.com/jhub/user/itoufiqu/tree>? ( > where crsplab2.domain.com <http://crsplab2.domain.com> is > the URL to get HAproxy ) > > > You need to tell your backend app that it runs behind reverse > proxy with ssl termination and that it's domain/url > is https://crsplab2.domain.com > <http://crsplab2.domain.com/jhub/user/itoufiqu/tree>. How you > do > that depends on the backend app you are using but most of them > like apache2, tomcat etc. have specific configs that you can > find in their documentation. For example if your backend is > apache2 I bet you don't have the DomainName set in the config > in > which case it defaults to the host ip address. > > > You can also try: > > rspirep ^Location:\ http://(.*):80(.*) Location:\ > https://crsplab2.domain.com > <http://crsplab2.domain.com/jhub/user/itoufiqu/tree>:443\2 if { > ssl_fc } > > to fix the URL but note that this will not save you from hard > coded > url's in the returned html pages the way apache does. > > > > While I am no expert in HA proxy world, I think this might > due to the fact that my backend does not have SSL and > HAproxy frontend does have SSL. At this point, I would > avoid that IP address showing up in the browser. what is > the best way to accomplish this? > > thanks for your continues help! > > > > > > > On Tue, Oct 23, 2018 at 8:35 AM Aleksandar Lazic > <[email protected] <mailto:[email protected]>> wrote: > > Hi. > > Am 23.10.2018 um 09:04 schrieb Imam Toufique: > > I am looking for some help on how to write the > following apache proxypass rules > > in HAproxy. Not to mention I am at a bit of loss > with > my first try :-) . Here > > are my current proxypass rules: > > > > ProxyPass http://10.1.100.156:8000/jhub > > ProxyPassReverse http://10.1.100.156:8000/jhub > > Well ProxyPass and ProxyPassReverse do a lot of thinks > not just rewrites, as > mentioned in the doc > > > https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass > > https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypassreverse > > > > <LocationMatch > > "/jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)"> > > ProxyPassMatch > ws://10.1.100.156:8000/jhub/$1/$2$3 > <http://10.1.100.156:8000/jhub/$1/$2$3> > > ProxyPassReverse > ws://10.1.100.156:8000/jhub/$1/$2$3 > <http://10.1.100.156:8000/jhub/$1/$2$3> > > </LocationMatch> > > > > As I am not well versed in the massive HAproxy > configuration guide, if any of > > you can give me a hand with this, I would very much > appreciate it. > > I'm also not "that" expert but I would try the > following, untested. > > ### > defaults > mode http > log global > > #... maybe some other settings > timeout tunnel 10h > > frontend https_001 > > #... maybe some other settings > > acl websocket path_beg /jhub > > #... maybe some other acls > > use_backend websocket_001 if websocket > > backend websocket_001 > > reqrep "^([^\ :]*) > > /jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)" > "/jhub/\1/\2\3" > > # You will need to replace the first column with the > response from the > # backend response > # rspirep "^Location: > > /jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)" > "Location: > /jhub/\1/\2\3" > # OR > # http-response replace-header Location > > "/jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)" > "/jhub/\1/\2\3" > > # add some checks > > server ws_01 10.1.100.156:8000 > <http://10.1.100.156:8000> check > ### > > Here are some links which may help you also. > > > https://www.haproxy.com/blog/websockets-load-balancing-with-haproxy/ > > https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-reqirep > > https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-rspirep > > I would run haproxy in Debug mode and see how the > request pass haproxy and adopt > the config. > > It would be nice when you show us the working conf ;-) > > It would be nice to have a > > http-request replace-uri <match-regex> <replace-fmt> > > to replace the reqrep. >
