Public bug reported: Description =========== I attempted to configure TLS between nova-spicehtml5proxy and Openstack nova KVM hypervisor. after enabling require_secure option and giving the paths to SSL related information on nova.conf, nova-spice5htmlproxy still connects to non-ssl port.
Steps to reproduce ================== on nova [spice] section set require_secure = True on compute set according your environment the following variables: server_proxyclient_address = html5proxy_base_url = html5proxy_host = html5proxy_port = server_listen = on nova-server set with your environment specific values following: server_listen = server_proxyclient_address = html5proxy_host = html5proxy_port = on [DEFAULT] section set values for key = <ssl_cert_key> cert = <ssl_cert> ssl_only = true both on compute hypervisor and nova-api server. configure /etc/libvirt/qemu.conf and set values: spice_tls = 1 spice_tls_x509_cert_dir = "/etc/pki/libvirt-spice" restart nova-api, nova-spice5html proxy on api server restart libvirtd and nova-compute on the hypervisor create a server on the specific hypervisor: openstack server create --image cirros --flavor m1.mini --network myipv4 --host node14.openstack.host test-spice-tls --os-compute-api-version 2.74 Login to your horizon instance, and go to instances -> console. We get just a gray spice window instead of console. Expected result =============== Spice console to the VM Actual result ============= A grey window without spice console Environment =========== Server environment: ii nova-api 2:30.0.0-5~bpo12+1 all OpenStack Compute - compute API frontend ii nova-common 2:30.0.0-5~bpo12+1 all OpenStack Compute - common files ii nova-conductor 2:30.0.0-5~bpo12+1 all OpenStack Compute - conductor service ii nova-consoleproxy 2:30.0.0-5~bpo12+1 all OpenStack Compute - NoVNC proxy ii nova-scheduler 2:30.0.0-5~bpo12+1 all OpenStack Compute - virtual machine scheduler ii python3-nova 2:30.0.0-5~bpo12+1 all OpenStack Compute - libraries ii python3-novaclient 2:18.7.0-3~bpo12+1 all client library for OpenStack Compute API - 3.x hypervisor: ii nova-common 2:30.0.0-5~bpo12+1 all OpenStack Compute - common files ii nova-compute 2:30.0.0-5~bpo12+1 all OpenStack Compute - compute node ii nova-compute-kvm 2:30.0.0-5~bpo12+1 all OpenStack Compute - compute node (KVM) ii python3-nova 2:30.0.0-5~bpo12+1 all OpenStack Compute - libraries ii python3-novaclient 2:18.7.0-3~bpo12+1 all client library for OpenStack Compute API - 3.x 2. Which hypervisor did you use? Libvirt + KVM dpkg -l|grep qemu ii ipxe-qemu 1.0.0+git-20190125.36a4c85-5.1 all PXE boot firmware - ROM images for qemu ii libvirt-daemon-driver-qemu 9.0.0-4+deb12u2 amd64 Virtualization daemon QEMU connection driver ii qemu-block-extra 1:7.2+dfsg-7+deb12u7 amd64 extra block backend modules for qemu-system and qemu-utils ii qemu-system-common 1:7.2+dfsg-7+deb12u7 amd64 QEMU full system emulation binaries (common files) ii qemu-system-data 1:7.2+dfsg-7+deb12u7 all QEMU full system emulation (data files) ii qemu-system-x86 1:7.2+dfsg-7+deb12u7 amd64 QEMU full system emulation binaries (x86) ii qemu-utils 1:7.2+dfsg-7+deb12u7 amd64 QEMU utilities dpkg -l|grep libvirt ii libvirt-clients 9.0.0-4+deb12u2 amd64 Programs for the libvirt library ii libvirt-daemon 9.0.0-4+deb12u2 amd64 Virtualization daemon ii libvirt-daemon-config-network 9.0.0-4+deb12u2 all Libvirt daemon configuration files (default network) ii libvirt-daemon-config-nwfilter 9.0.0-4+deb12u2 all Libvirt daemon configuration files (default network filters) ii libvirt-daemon-driver-qemu 9.0.0-4+deb12u2 amd64 Virtualization daemon QEMU connection driver ii libvirt-daemon-driver-storage-rbd 9.0.0-4+deb12u2 amd64 Virtualization daemon RBD storage driver ii libvirt-daemon-system 9.0.0-4+deb12u2 amd64 Libvirt daemon configuration files ii libvirt-daemon-system-systemd 9.0.0-4+deb12u2 all Libvirt daemon configuration files (systemd) ii libvirt0:amd64 9.0.0-4+deb12u2 amd64 library for interfacing with different virtualization systems ii python3-libvirt 10.0.0-1~bpo12+1 amd64 libvirt Python 3 bindings 2. Which storage type did you use? Ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable) 3. Which networking type did you use? Neutron with OVN Logs & Configs ============== nova-api/spice5html-proxy [DEFAULT] ... key = /srv/cert/privkey.pem cert = /srv/cert/cert.pem ssl_only = true ... [spice] enabled = true require_secure = true agent_enabled = False source_is_ipv6 = True server_listen = <public ip> server_proxyclient_address = <public ip> html5proxy_host = <public ip> html5proxy_port = 6082 ... Open ports when running spice5html ss -tulpn Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process udp UNCONN 0 0 127.0.0.54:53 0.0.0.0:* users:(("systemd-resolve",pid=84,fd=16)) udp UNCONN 0 0 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=84,fd=14)) tcp LISTEN 0 100 0.0.0.0:25 0.0.0.0:* users:(("master",pid=465,fd=13)) tcp LISTEN 0 4096 127.0.0.54:53 0.0.0.0:* users:(("systemd-resolve",pid=84,fd=17)) tcp LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=84,fd=15)) tcp LISTEN 0 511 *:8775 *:* users:(("apache2",pid=66907,fd=6),("apache2",pid=66906,fd=6),("apache2",pid=66890,fd=6)) tcp LISTEN 0 511 *:8774 *:* users:(("apache2",pid=66907,fd=5),("apache2",pid=66906,fd=5),("apache2",pid=66890,fd=5)) tcp LISTEN 0 511 *:80 *:* users:(("apache2",pid=66907,fd=3),("apache2",pid=66906,fd=3),("apache2",pid=66890,fd=3)) tcp LISTEN 0 4096 *:22 *:* users:(("sshd",pid=97,fd=3),("systemd",pid=1,fd=78)) tcp LISTEN 0 100 [::]:25 [::]:* users:(("master",pid=465,fd=14)) tcp LISTEN 0 511 *:443 *:* users:(("apache2",pid=66907,fd=4),("apache2",pid=66906,fd=4),("apache2",pid=66890,fd=4)) tcp LISTEN 0 100 [public_ipv6]:6082 [::]:* users:(("nova-spicehtml5",pid=66877,fd=10)) Compute node configuration [DEFAULT] debug = true my_ip = <my_ipv6_internal> pybasedir = /usr/lib/python3/dist-packages state_path = /var/lib/nova transport_url = rabbit://openstack:rabbitpass@rabbitmq.internal.cloud:5671// key = /srv/hostname/certs/privkey.pem cert = /srv/hostname/certs/cert.pem ssl_only = True ... [spice] source_is_ipv6 = true enabled = true agent_enabled = False require_secure = True server_proxyclient_address = $my_ip html5proxy_base_url = https:/<proxy public ip>:6082/spice_auto.html html5proxy_host = "<proxy public ip>" html5proxy_port = 6082 server_listen = $my_ip ... /etc/libvirt/qemu.conf spice_tls = 1 spice_tls_x509_cert_dir = "/etc/pki/libvirt-spice" virtual machine XML console part after creation: <graphics type='spice' port='5900' tlsPort='5901' autoport='yes' listen='$my_ip'> <listen type='address' address='$my_ip'/> <channel name='main' mode='secure'/> <channel name='display' mode='secure'/> <channel name='inputs' mode='secure'/> <channel name='cursor' mode='secure'/> <channel name='playback' mode='secure'/> <channel name='record' mode='secure'/> <channel name='smartcard' mode='secure'/> <channel name='usbredir' mode='secure'/> </graphics> tcpdump on server when trying to access spice console on horizon: tcpdump -i eno1 dst node14.internal.openstack.cloud tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on eno1, link-type EN10MB (Ethernet), snapshot length 262144 bytes 12:25:13.095419 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [S], seq 3792735556, win 65535, options [mss 1440,sackOK,TS val 3475825695 ecr 0,nop,wscale 8], length 0 12:25:13.095700 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [.], ack 1127036222, win 19530, options [nop,nop,TS val 3475825696 ecr 835323250], length 0 12:25:13.097215 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [P.], seq 0:42, ack 1, win 19530, options [nop,nop,TS val 3475825697 ecr 835323250], length 42 12:25:13.097582 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [.], ack 195, win 19530, options [nop,nop,TS val 3475825698 ecr 835323252], length 0 12:25:13.098171 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [F.], seq 42, ack 196, win 19530, options [nop,nop,TS val 3475825698 ecr 835323252], length 0 12:25:54.221794 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [S], seq 2638634823, win 65535, options [mss 1440,sackOK,TS val 3475866822 ecr 0,nop,wscale 8], length 0 12:25:54.222118 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [.], ack 2255119891, win 19530, options [nop,nop,TS val 3475866822 ecr 835364376], length 0 12:25:54.223687 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [P.], seq 0:42, ack 1, win 19530, options [nop,nop,TS val 3475866824 ecr 835364376], length 42 12:25:54.223997 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [.], ack 195, win 19530, options [nop,nop,TS val 3475866824 ecr 835364378], length 0 12:25:54.224505 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [F.], seq 42, ack 196, win 19530, options [nop,nop,TS val 3475866824 ecr 835364378], length 0 on nova hypervisor: tcpdump -i cloud-int src nova.internal.openstack.cloud tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on cloud-int, link-type EN10MB (Ethernet), snapshot length 262144 bytes 12:25:54.221632 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [S], seq 2638634823, win 65535, options [mss 1440,sackOK,TS val 3475866822 ecr 0,nop,wscale 8], length 0 12:25:54.221888 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [.], ack 2255119891, win 19530, options [nop,nop,TS val 3475866822 ecr 835364376], length 0 12:25:54.223502 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [P.], seq 0:42, ack 1, win 19530, options [nop,nop,TS val 3475866824 ecr 835364376], length 42 12:25:54.223760 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [.], ack 195, win 19530, options [nop,nop,TS val 3475866824 ecr 835364378], length 0 12:25:54.224261 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [F.], seq 42, ack 196, win 19530, options [nop,nop,TS val 3475866824 ecr 835364378], length 0 Traffic should go to 5901, not 5900! We run nova-api with apache2 wsgi, config here: Listen 8774 Listen 8775 <VirtualHost *:8774> SSLEngine on SSLHonorCipherOrder on SSLCertificateFile /srv/nova.openstack.cloud/dehydrated/certs/nova.openstack.cloud/fullchain.pem SSLCertificateKeyFile /srv/nova.openstack.cloud/dehydrated/certs/nova.openstack.cloud/privkey.pem WSGIScriptAlias / /usr/bin/nova-api-wsgi WSGIDaemonProcess nova-api processes=5 threads=1 user=nova group=nova display-name=%{GROUP} WSGIProcessGroup nova-api WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On LimitRequestBody 114688 <IfVersion >= 2.4> ErrorLogFormat "%{cu}t %M" </IfVersion> ErrorLog /var/log/apache2/nova-api-error.log CustomLog /var/log/apache2/nova-api-access.log combined <Directory /usr/bin> Require all granted </Directory> </VirtualHost> Alias /compute /usr/bin/nova-api-wsgi <Location /compute> SetHandler wsgi-script Options +ExecCGI WSGIProcessGroup nova-api WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On </Location> <VirtualHost *:8775> SSLEngine on SSLHonorCipherOrder on SSLCertificateFile /srv/nova.openstack.cloud/dehydrated/certs/nova.openstack.cloud/fullchain.pem SSLCertificateKeyFile /srv/nova.openstack.cloud/dehydrated/certs/nova.openstack.cloud/privkey.pem WSGIScriptAlias / /usr/bin/nova-metadata-wsgi WSGIDaemonProcess nova-api-metadata processes=5 threads=1 user=nova group=nova display-name=%{GROUP} WSGIProcessGroup nova-api-metadata WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On LimitRequestBody 114688 <IfVersion >= 2.4> ErrorLogFormat "%{cu}t %M" </IfVersion> ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2103513 Title: Spice console TLS doesnt work as expected Status in OpenStack Compute (nova): New Bug description: Description =========== I attempted to configure TLS between nova-spicehtml5proxy and Openstack nova KVM hypervisor. after enabling require_secure option and giving the paths to SSL related information on nova.conf, nova-spice5htmlproxy still connects to non-ssl port. Steps to reproduce ================== on nova [spice] section set require_secure = True on compute set according your environment the following variables: server_proxyclient_address = html5proxy_base_url = html5proxy_host = html5proxy_port = server_listen = on nova-server set with your environment specific values following: server_listen = server_proxyclient_address = html5proxy_host = html5proxy_port = on [DEFAULT] section set values for key = <ssl_cert_key> cert = <ssl_cert> ssl_only = true both on compute hypervisor and nova-api server. configure /etc/libvirt/qemu.conf and set values: spice_tls = 1 spice_tls_x509_cert_dir = "/etc/pki/libvirt-spice" restart nova-api, nova-spice5html proxy on api server restart libvirtd and nova-compute on the hypervisor create a server on the specific hypervisor: openstack server create --image cirros --flavor m1.mini --network myipv4 --host node14.openstack.host test-spice-tls --os-compute-api-version 2.74 Login to your horizon instance, and go to instances -> console. We get just a gray spice window instead of console. Expected result =============== Spice console to the VM Actual result ============= A grey window without spice console Environment =========== Server environment: ii nova-api 2:30.0.0-5~bpo12+1 all OpenStack Compute - compute API frontend ii nova-common 2:30.0.0-5~bpo12+1 all OpenStack Compute - common files ii nova-conductor 2:30.0.0-5~bpo12+1 all OpenStack Compute - conductor service ii nova-consoleproxy 2:30.0.0-5~bpo12+1 all OpenStack Compute - NoVNC proxy ii nova-scheduler 2:30.0.0-5~bpo12+1 all OpenStack Compute - virtual machine scheduler ii python3-nova 2:30.0.0-5~bpo12+1 all OpenStack Compute - libraries ii python3-novaclient 2:18.7.0-3~bpo12+1 all client library for OpenStack Compute API - 3.x hypervisor: ii nova-common 2:30.0.0-5~bpo12+1 all OpenStack Compute - common files ii nova-compute 2:30.0.0-5~bpo12+1 all OpenStack Compute - compute node ii nova-compute-kvm 2:30.0.0-5~bpo12+1 all OpenStack Compute - compute node (KVM) ii python3-nova 2:30.0.0-5~bpo12+1 all OpenStack Compute - libraries ii python3-novaclient 2:18.7.0-3~bpo12+1 all client library for OpenStack Compute API - 3.x 2. Which hypervisor did you use? Libvirt + KVM dpkg -l|grep qemu ii ipxe-qemu 1.0.0+git-20190125.36a4c85-5.1 all PXE boot firmware - ROM images for qemu ii libvirt-daemon-driver-qemu 9.0.0-4+deb12u2 amd64 Virtualization daemon QEMU connection driver ii qemu-block-extra 1:7.2+dfsg-7+deb12u7 amd64 extra block backend modules for qemu-system and qemu-utils ii qemu-system-common 1:7.2+dfsg-7+deb12u7 amd64 QEMU full system emulation binaries (common files) ii qemu-system-data 1:7.2+dfsg-7+deb12u7 all QEMU full system emulation (data files) ii qemu-system-x86 1:7.2+dfsg-7+deb12u7 amd64 QEMU full system emulation binaries (x86) ii qemu-utils 1:7.2+dfsg-7+deb12u7 amd64 QEMU utilities dpkg -l|grep libvirt ii libvirt-clients 9.0.0-4+deb12u2 amd64 Programs for the libvirt library ii libvirt-daemon 9.0.0-4+deb12u2 amd64 Virtualization daemon ii libvirt-daemon-config-network 9.0.0-4+deb12u2 all Libvirt daemon configuration files (default network) ii libvirt-daemon-config-nwfilter 9.0.0-4+deb12u2 all Libvirt daemon configuration files (default network filters) ii libvirt-daemon-driver-qemu 9.0.0-4+deb12u2 amd64 Virtualization daemon QEMU connection driver ii libvirt-daemon-driver-storage-rbd 9.0.0-4+deb12u2 amd64 Virtualization daemon RBD storage driver ii libvirt-daemon-system 9.0.0-4+deb12u2 amd64 Libvirt daemon configuration files ii libvirt-daemon-system-systemd 9.0.0-4+deb12u2 all Libvirt daemon configuration files (systemd) ii libvirt0:amd64 9.0.0-4+deb12u2 amd64 library for interfacing with different virtualization systems ii python3-libvirt 10.0.0-1~bpo12+1 amd64 libvirt Python 3 bindings 2. Which storage type did you use? Ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable) 3. Which networking type did you use? Neutron with OVN Logs & Configs ============== nova-api/spice5html-proxy [DEFAULT] ... key = /srv/cert/privkey.pem cert = /srv/cert/cert.pem ssl_only = true ... [spice] enabled = true require_secure = true agent_enabled = False source_is_ipv6 = True server_listen = <public ip> server_proxyclient_address = <public ip> html5proxy_host = <public ip> html5proxy_port = 6082 ... Open ports when running spice5html ss -tulpn Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process udp UNCONN 0 0 127.0.0.54:53 0.0.0.0:* users:(("systemd-resolve",pid=84,fd=16)) udp UNCONN 0 0 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=84,fd=14)) tcp LISTEN 0 100 0.0.0.0:25 0.0.0.0:* users:(("master",pid=465,fd=13)) tcp LISTEN 0 4096 127.0.0.54:53 0.0.0.0:* users:(("systemd-resolve",pid=84,fd=17)) tcp LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=84,fd=15)) tcp LISTEN 0 511 *:8775 *:* users:(("apache2",pid=66907,fd=6),("apache2",pid=66906,fd=6),("apache2",pid=66890,fd=6)) tcp LISTEN 0 511 *:8774 *:* users:(("apache2",pid=66907,fd=5),("apache2",pid=66906,fd=5),("apache2",pid=66890,fd=5)) tcp LISTEN 0 511 *:80 *:* users:(("apache2",pid=66907,fd=3),("apache2",pid=66906,fd=3),("apache2",pid=66890,fd=3)) tcp LISTEN 0 4096 *:22 *:* users:(("sshd",pid=97,fd=3),("systemd",pid=1,fd=78)) tcp LISTEN 0 100 [::]:25 [::]:* users:(("master",pid=465,fd=14)) tcp LISTEN 0 511 *:443 *:* users:(("apache2",pid=66907,fd=4),("apache2",pid=66906,fd=4),("apache2",pid=66890,fd=4)) tcp LISTEN 0 100 [public_ipv6]:6082 [::]:* users:(("nova-spicehtml5",pid=66877,fd=10)) Compute node configuration [DEFAULT] debug = true my_ip = <my_ipv6_internal> pybasedir = /usr/lib/python3/dist-packages state_path = /var/lib/nova transport_url = rabbit://openstack:rabbitpass@rabbitmq.internal.cloud:5671// key = /srv/hostname/certs/privkey.pem cert = /srv/hostname/certs/cert.pem ssl_only = True ... [spice] source_is_ipv6 = true enabled = true agent_enabled = False require_secure = True server_proxyclient_address = $my_ip html5proxy_base_url = https:/<proxy public ip>:6082/spice_auto.html html5proxy_host = "<proxy public ip>" html5proxy_port = 6082 server_listen = $my_ip ... /etc/libvirt/qemu.conf spice_tls = 1 spice_tls_x509_cert_dir = "/etc/pki/libvirt-spice" virtual machine XML console part after creation: <graphics type='spice' port='5900' tlsPort='5901' autoport='yes' listen='$my_ip'> <listen type='address' address='$my_ip'/> <channel name='main' mode='secure'/> <channel name='display' mode='secure'/> <channel name='inputs' mode='secure'/> <channel name='cursor' mode='secure'/> <channel name='playback' mode='secure'/> <channel name='record' mode='secure'/> <channel name='smartcard' mode='secure'/> <channel name='usbredir' mode='secure'/> </graphics> tcpdump on server when trying to access spice console on horizon: tcpdump -i eno1 dst node14.internal.openstack.cloud tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on eno1, link-type EN10MB (Ethernet), snapshot length 262144 bytes 12:25:13.095419 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [S], seq 3792735556, win 65535, options [mss 1440,sackOK,TS val 3475825695 ecr 0,nop,wscale 8], length 0 12:25:13.095700 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [.], ack 1127036222, win 19530, options [nop,nop,TS val 3475825696 ecr 835323250], length 0 12:25:13.097215 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [P.], seq 0:42, ack 1, win 19530, options [nop,nop,TS val 3475825697 ecr 835323250], length 42 12:25:13.097582 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [.], ack 195, win 19530, options [nop,nop,TS val 3475825698 ecr 835323252], length 0 12:25:13.098171 IP6 nova.internal.openstack.cloud.50286 > node14.internal.openstack.cloud.5900: Flags [F.], seq 42, ack 196, win 19530, options [nop,nop,TS val 3475825698 ecr 835323252], length 0 12:25:54.221794 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [S], seq 2638634823, win 65535, options [mss 1440,sackOK,TS val 3475866822 ecr 0,nop,wscale 8], length 0 12:25:54.222118 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [.], ack 2255119891, win 19530, options [nop,nop,TS val 3475866822 ecr 835364376], length 0 12:25:54.223687 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [P.], seq 0:42, ack 1, win 19530, options [nop,nop,TS val 3475866824 ecr 835364376], length 42 12:25:54.223997 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [.], ack 195, win 19530, options [nop,nop,TS val 3475866824 ecr 835364378], length 0 12:25:54.224505 IP6 nova.internal.openstack.cloud.39964 > node14.internal.openstack.cloud.5900: Flags [F.], seq 42, ack 196, win 19530, options [nop,nop,TS val 3475866824 ecr 835364378], length 0 on nova hypervisor: tcpdump -i cloud-int src nova.internal.openstack.cloud tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on cloud-int, link-type EN10MB (Ethernet), snapshot length 262144 bytes 12:25:54.221632 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [S], seq 2638634823, win 65535, options [mss 1440,sackOK,TS val 3475866822 ecr 0,nop,wscale 8], length 0 12:25:54.221888 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [.], ack 2255119891, win 19530, options [nop,nop,TS val 3475866822 ecr 835364376], length 0 12:25:54.223502 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [P.], seq 0:42, ack 1, win 19530, options [nop,nop,TS val 3475866824 ecr 835364376], length 42 12:25:54.223760 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [.], ack 195, win 19530, options [nop,nop,TS val 3475866824 ecr 835364378], length 0 12:25:54.224261 IP6 nova.internal.openstack..cloud.39964 > node14.internal.openstack.cloud.5900: Flags [F.], seq 42, ack 196, win 19530, options [nop,nop,TS val 3475866824 ecr 835364378], length 0 Traffic should go to 5901, not 5900! We run nova-api with apache2 wsgi, config here: Listen 8774 Listen 8775 <VirtualHost *:8774> SSLEngine on SSLHonorCipherOrder on SSLCertificateFile /srv/nova.openstack.cloud/dehydrated/certs/nova.openstack.cloud/fullchain.pem SSLCertificateKeyFile /srv/nova.openstack.cloud/dehydrated/certs/nova.openstack.cloud/privkey.pem WSGIScriptAlias / /usr/bin/nova-api-wsgi WSGIDaemonProcess nova-api processes=5 threads=1 user=nova group=nova display-name=%{GROUP} WSGIProcessGroup nova-api WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On LimitRequestBody 114688 <IfVersion >= 2.4> ErrorLogFormat "%{cu}t %M" </IfVersion> ErrorLog /var/log/apache2/nova-api-error.log CustomLog /var/log/apache2/nova-api-access.log combined <Directory /usr/bin> Require all granted </Directory> </VirtualHost> Alias /compute /usr/bin/nova-api-wsgi <Location /compute> SetHandler wsgi-script Options +ExecCGI WSGIProcessGroup nova-api WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On </Location> <VirtualHost *:8775> SSLEngine on SSLHonorCipherOrder on SSLCertificateFile /srv/nova.openstack.cloud/dehydrated/certs/nova.openstack.cloud/fullchain.pem SSLCertificateKeyFile /srv/nova.openstack.cloud/dehydrated/certs/nova.openstack.cloud/privkey.pem WSGIScriptAlias / /usr/bin/nova-metadata-wsgi WSGIDaemonProcess nova-api-metadata processes=5 threads=1 user=nova group=nova display-name=%{GROUP} WSGIProcessGroup nova-api-metadata WSGIApplicationGroup %{GLOBAL} WSGIPassAuthorization On LimitRequestBody 114688 <IfVersion >= 2.4> ErrorLogFormat "%{cu}t %M" </IfVersion> To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2103513/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp