Hi Igor,
Thanks very much for offering to help! I will do this in sections,
hopefully, I can keep this from being too cluttered.
haproxy.cfg:
--------------------------------------------------------------------------------------
global
#log /dev/log local0 debug
#log /dev/log local1 debug
log 127.0.0.1 local2
chroot /var/lib/haproxy
stats timeout 30s
user haproxy
group haproxy
tune.ssl.default-dh-param 2048
daemon
defaults
log global
mode http
option tcplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
timeout tunnel 9h
option tcp-check
frontend http_front
bind :80
bind 0.0.0.0:443 ssl crt /etc/haproxy/crsplab2_1.pem
stats uri /haproxy?stats
default_backend web1_cluster
option httplog
log global
#option dontlognull
log /dev/log local0 debug
mode http
option forwardfor # forward IP
http-request set-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto https if { ssl_fc }
redirect scheme https if !{ ssl_fc }
acl host_web2 hdr(host) -i crsplab2.oit.uci.edu/webdav
use_backend webdav_cluster if host_web2
acl host_web3 path_beg /jhub
use_backend web3_cluster if host_web3
backend webdav_cluster
balance roundrobin
server web1 10.1.100.156:8080 check inter 2000 cookie w1
server web2 10.1.100.160:8080 check inter 2000 cookie w2
backend web3_cluster
server publicIP:443 check ssl verify none inter 2000 cookie w1
-----------------------------------------------------------------------------------------------------
Note: I have a single backend node, as it was easy to test with just one
node, instead of making changes to 2 nodes at a time.
Here is my apache config:
in httpd.conf, only change I have made is ( the rest is a stock centos 7.5
httpd.conf ):
-------------------------------------
ServerName 10.1.100.160:80 ( Internal IP of the backend node)
Redirect permanent /jhub https://crsplabweb1.domain.com/jhub
-------------------------------------
in my ssl.conf, where I access the jupyterhub instance running in
127.0.0.1:8000 . Also, note that the backend is running shibboleth SP.
One of the issues I encountered is, If I did not have SSL , i was getting a
browser warning for not having SSL.
Here is my ssl.conf:
--------------------------------------------------------------------------
Listen 443 https
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog
SSLSessionCache shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout 300
SSLRandomSeed startup file:/dev/urandom 256
SSLRandomSeed connect builtin
SSLCryptoDevice builtin
<VirtualHost _default_:443>
UseCanonicalName on
ServerName crsplabweb1.domain.com:443
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
LogLevel warn
SSLEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite HIGH:3DES:!aNULL:!MD5:!SEED:!IDEA
SSLCertificateFile /etc/pki/tls/certs/crsplabweb1.domain.com_cert.cer
SSLCertificateKeyFile /etc/pki/tls/certs/crsplabweb2.key
SSLCertificateChainFile
/etc/pki/tls/certs/crsplabweb1.domain.com_interm_reverse.c
<Files ~ "\.(cgi|shtml|phtml|php3?)$">
SSLOptions +StdEnvVars
</Files>
<Directory "/var/www/cgi-bin">
SSLOptions +StdEnvVars
</Directory>
<Location /jhub>
ProxyPass http://127.0.0.1:8000/jhub
ProxyPassReverse http://127.0.0.1:8000/jhub
RequestHeader unset Accept-Encoding
ProxyPreserveHost on
AuthType shibboleth
ShibRequestSetting requireSession 1
Require shibboleth
ShibUseHeaders On
ShibBasicHijack On
RewriteEngine On
RequestHeader set X-Remote-User %{REMOTE_USER}s
</Location>
<LocationMatch
"/jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)">
ProxyPassMatch ws://127.0.0.1:8000/jhub/$1/$2$3
ProxyPassReverse ws://127.0.0.1:8000/jhub/$1/$2$3
</LocationMatch>
BrowserMatch "MSIE [2-5]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
CustomLog logs/ssl_request_log \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
</VirtualHost>
----------------------------------------------------------------------------------
Thanks
On Fri, Oct 26, 2018 at 8:34 PM Igor Cicimov <[email protected]>
wrote:
> Hi Imam,
>
> On Sat, Oct 27, 2018 at 9:37 AM Imam Toufique <[email protected]> wrote:
>
>> Hi,
>>
>> I came up with the following config, things seem to be working now, for
>> the most part.
>>
>> frontend http_front
>> bind :80
>> bind 0.0.0.0:443 ssl crt /etc/haproxy/crsplab2_1.pem
>> stats uri /haproxy?stats
>> default_backend web1_cluster
>> option httplog
>> log global
>> #option dontlognull
>> log /dev/log local0 debug
>> mode http
>> option forwardfor # forward IP
>> http-request set-header X-Forwarded-Port %[dst_port]
>> http-request add-header X-Forwarded-Proto https if { ssl_fc }
>> redirect scheme https if !{ ssl_fc }
>> acl host_web3 path_beg /jhub
>> use_backend web3_cluster if host_web3
>>
>> web3_cluster
>>
>> backend web3_cluster
>> mode http
>> balance source
>> server crsplabweb1.domain.com publicIP:443 check ssl verify none
>> inter 2000 cookie w1
>>
>> The above config gets me to the backend node -- where I have a
>> jupyterhub instance running + . Shibboleth SP running for authentication.
>> As I could not get shibboleth SP to work by staying in my private network,
>> I had to set up a public IP for the backend node, get SSL certs - so
>> shibboleth authentication could be done. I am sure there is a better
>> approach to this, but I don't know what it is. I will be trying out SNAT
>> to see if that will allow me to keep using my private IP for the backend
>> nodes. If any of you know how to do SNAT, please chime in, it would be
>> worth the time/effort to try it out.
>>
>> Now, the interesting thing I have noticed with the above setup -- when I
>> connect to HAProxy, let's say with https://proxy.domain.com , I
>> authenticate with shibboleth, and then the URL in the browser points to the
>> backend node.
>>
>> For example:
>>
>> my proxy address: https://proxy.domain.com/jhub
>>
>> after I connect to the backend, the URL turns into -
>> https://crsplabweb1.domain.com/jhub/tree?
>>
>> ...and everything works thereafter.
>>
>> I tried the rewrite method that Igor has suggested before, that did not
>> make any difference. But what I noticed is, after I connect, no traffic go
>> through the proxy anymore, my client ( i.e. laptop) connects directly to
>> the backend server. Not sure if this good or bad though (?) , but, I am not
>> sure how to configure this so that I will go through a proxy but still be
>> connected in the backend via a private IP and I can ( still ) authenticate
>> via shibboleth.
>>
>> So, when I change the 'web3_cluster' backend to :
>>
>> server crsplabweb1 privateIP:80 inter 2000 cookie w1
>>
>> and, I set backend apache to accept connection on port 80, then I break
>> shibboleth authentication.
>>
>> Any inputs here?
>>
>> thanks, guys!
>>
>>
> I think it is time for you to provide the full HAP and Apache configs so
> we can see what is going on (please obfuscate any sensitive data). Also the
> use of the "cookie w1" is not clear since you are not setting it in HAP
> and is kinda redundant for single backend setup.
>
>
>>
>> On Thu, Oct 25, 2018 at 1:21 AM Igor Cicimov <
>> [email protected]> wrote:
>>
>>>
>>>
>>> On Thu, Oct 25, 2018 at 6:31 PM Igor Cicimov <
>>> [email protected]> wrote:
>>>
>>>>
>>>>
>>>> On Thu, 25 Oct 2018 6:13 pm Imam Toufique <[email protected]> wrote:
>>>>
>>>>> so I almost got this to work, based on the situation I am in. To
>>>>> elaborate just a bit, my setup involves a shibboleth SP that I need to
>>>>> authenticate my application. Since I can't set up the HA proxy node with
>>>>> shibboleth SP - I had to wrap my application in the backend with apache so
>>>>> I can pass REMOTE_USER to the application. the application I have is -
>>>>> jupyterhub and it start with its own proxy. Long story short, here is my
>>>>> current setup:
>>>>>
>>>>> frontend
>>>>> bind :80
>>>>> bind :443 ssl crt /etc/haproxy/crsplab2_1.pem
>>>>> stats uri /haproxy?stats
>>>>> default_backend web1_cluster
>>>>> option httplog
>>>>> log global
>>>>> #option dontlognull
>>>>> log /dev/log local0 debug
>>>>> mode http
>>>>> option forwardfor # forward IP
>>>>> http-request set-header X-Forwarded-Port %[dst_port]
>>>>> http-request add-header X-Forwarded-Proto https if { ssl_fc }
>>>>> redirect scheme https if !{ ssl_fc }
>>>>>
>>>>> acl host_web3 path_beg /jhub
>>>>> use_backend web3_cluster if host_web3
>>>>>
>>>>> backend
>>>>> server web1.oit.uci.edu 128.110.80.5:80 check
>>>>>
>>>>> this works for the most part. But I am confused with a problem. when
>>>>> I get to my application, my backend IP address shows up in the browser
>>>>> URL.
>>>>>
>>>>> for example, I see this in my browser:
>>>>>
>>>>> http://128.110.80.5/jhub/user/itoufiqu/tree?
>>>>>
>>>>> whereas, I was expecting that it would show the original URL, such as:
>>>>>
>>>>> http://crsplab2.domain.com/jhub/user/itoufiqu/tree? ( where
>>>>> crsplab2.domain.com is the URL to get HAproxy )
>>>>>
>>>>
>>>> You need to tell your backend app that it runs behind reverse proxy
>>>> with ssl termination and that it's domain/url is
>>>> https://crsplab2.domain.com
>>>> <http://crsplab2.domain.com/jhub/user/itoufiqu/tree>. How you do that
>>>> depends on the backend app you are using but most of them like apache2,
>>>> tomcat etc. have specific configs that you can find in their documentation.
>>>> For example if your backend is apache2 I bet you don't have the DomainName
>>>> set in the config in which case it defaults to the host ip address.
>>>>
>>>
>>> You can also try:
>>>
>>> rspirep ^Location:\ http://(.*):80(.*) Location:\ https://
>>> crsplab2.domain.com
>>> <http://crsplab2.domain.com/jhub/user/itoufiqu/tree>:443\2
>>> if { ssl_fc }
>>>
>>> to fix the URL but note that this will not save you from hard coded
>>> url's in the returned html pages the way apache does.
>>>
>>>
>>>>
>>>>> While I am no expert in HA proxy world, I think this might due to the
>>>>> fact that my backend does not have SSL and HAproxy frontend does have SSL.
>>>>> At this point, I would avoid that IP address showing up in the browser.
>>>>> what is the best way to accomplish this?
>>>>>
>>>>> thanks for your continues help!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Oct 23, 2018 at 8:35 AM Aleksandar Lazic <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> Hi.
>>>>>>
>>>>>> Am 23.10.2018 um 09:04 schrieb Imam Toufique:
>>>>>> > I am looking for some help on how to write the following apache
>>>>>> proxypass rules
>>>>>> > in HAproxy. Not to mention I am at a bit of loss with my first try
>>>>>> :-) . Here
>>>>>> > are my current proxypass rules:
>>>>>> >
>>>>>> > ProxyPass http://10.1.100.156:8000/jhub
>>>>>> > ProxyPassReverse http://10.1.100.156:8000/jhub
>>>>>>
>>>>>> Well ProxyPass and ProxyPassReverse do a lot of thinks not just
>>>>>> rewrites, as
>>>>>> mentioned in the doc
>>>>>>
>>>>>> https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass
>>>>>> https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypassreverse
>>>>>>
>>>>>>
>>>>>> > <LocationMatch
>>>>>> "/jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)">
>>>>>> > ProxyPassMatch ws://10.1.100.156:8000/jhub/$1/$2$3
>>>>>> > ProxyPassReverse ws://10.1.100.156:8000/jhub/$1/$2$3
>>>>>> > </LocationMatch>
>>>>>> >
>>>>>> > As I am not well versed in the massive HAproxy configuration guide,
>>>>>> if any of
>>>>>> > you can give me a hand with this, I would very much appreciate it.
>>>>>>
>>>>>> I'm also not "that" expert but I would try the following, untested.
>>>>>>
>>>>>> ###
>>>>>> defaults
>>>>>> mode http
>>>>>> log global
>>>>>>
>>>>>> #... maybe some other settings
>>>>>> timeout tunnel 10h
>>>>>>
>>>>>> frontend https_001
>>>>>>
>>>>>> #... maybe some other settings
>>>>>>
>>>>>> acl websocket path_beg /jhub
>>>>>>
>>>>>> #... maybe some other acls
>>>>>>
>>>>>> use_backend websocket_001 if websocket
>>>>>>
>>>>>> backend websocket_001
>>>>>>
>>>>>> reqrep "^([^\ :]*)
>>>>>> /jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)"
>>>>>> "/jhub/\1/\2\3"
>>>>>>
>>>>>> # You will need to replace the first column with the response from
>>>>>> the
>>>>>> # backend response
>>>>>> # rspirep "^Location:
>>>>>> /jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)"
>>>>>> "Location:
>>>>>> /jhub/\1/\2\3"
>>>>>> # OR
>>>>>> # http-response replace-header Location
>>>>>> "/jhub/(user/[^/]*)/(api/kernels/[^/]+/channels/websocket)(.*)"
>>>>>> "/jhub/\1/\2\3"
>>>>>>
>>>>>> # add some checks
>>>>>>
>>>>>> server ws_01 10.1.100.156:8000 check
>>>>>> ###
>>>>>>
>>>>>> Here are some links which may help you also.
>>>>>>
>>>>>> https://www.haproxy.com/blog/websockets-load-balancing-with-haproxy/
>>>>>>
>>>>>> https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-reqirep
>>>>>>
>>>>>> https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-rspirep
>>>>>>
>>>>>> I would run haproxy in Debug mode and see how the request pass
>>>>>> haproxy and adopt
>>>>>> the config.
>>>>>>
>>>>>> It would be nice when you show us the working conf ;-)
>>>>>>
>>>>>> It would be nice to have a
>>>>>>
>>>>>> http-request replace-uri <match-regex> <replace-fmt>
>>>>>>
>>>>>> to replace the reqrep.
>>>>>>
>>>>>> > thanks
>>>>>>
>>>>>> Hth
>>>>>> Aleks
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> *Imam Toufique*
>>>>> *213-700-5485*
>>>>>
>>>>
>>>
>>> --
>>> Igor Cicimov | DevOps
>>>
>>>
>>> p. +61 (0) 433 078 728
>>> e. [email protected] <http://encompasscorporation.com/>
>>> w*.* www.encompasscorporation.com
>>> a. Level 4, 65 York Street, Sydney 2000
>>>
>>
>>
>> --
>> Regards,
>> *Imam Toufique*
>> *213-700-5485*
>>
>
>
> --
> Igor Cicimov | DevOps
>
>
> p. +61 (0) 433 078 728
> e. [email protected] <http://encompasscorporation.com/>
> w*.* www.encompasscorporation.com
> a. Level 4, 65 York Street, Sydney 2000
>
--
Regards,
*Imam Toufique*
*213-700-5485*
