Matt: I am looking into the issue now. I think you need to offload the SSL to Cloudflare with NGNIX Proxy. I am capturing screenshots of my setup to see if that will help you.
Doug Jenkins On Saturday, July 24, 2021 at 12:33:22 PM UTC-4 [email protected] wrote: > Doing more trouble shooting. I have narrowed it down to a SSL issue. > > To confirm this I disabled SSL at cloudflare and disabled force SSL for > shakerweather.com with my NGINX reverse proxy server and ran with this > setup: > > *weewx.config* > [[MQTT]] > server_url = mqtt://usr:[email protected]:1883/ > > topic = weather > unit_system = US > binding = archive, loop > aggregation = aggregate > > *skin.conf* > # MQTT Websockets defaults > mqtt_websockets_enabled = 1 > mqtt_websockets_host = "mqtt.shakerweather.com" > mqtt_websockets_port = 9001 > mqtt_websockets_ssl = 0 > mqtt_websockets_topic = "weather/loop" > disconnect_live_website_visitor = 1800000 > > *mosquitto.conf* > persistence false > > # mqtt > listener 1883 > protocol mqtt > > # websockets > listener 9001 > protocol websockets > > allow_anonymous true > password_file /etc/mosquitto/passwd > acl_file /etc/mosquitto/acl > > > I am able to see the real time websocket updates internally and externally > at http://shakerweather.com (I'll leave it up for a short time so if it > doesn't work if you click on it that could be why) > > So at least I have narrowed down the issue. Still not sure how to resolve > to ensure all is secured with SSL.... > On Saturday, July 24, 2021 at 10:23:03 AM UTC-4 Matt Johnson wrote: > >> Your setup seems similar to mine so I went for it. Still not working. >> >> I went ahead and added a mqtt CNAME record to the shakerweather.com >> cloudflare setup. I also added mqtt.shakerweather.com to my NGINX Proxy >> Server to forward to port 9001 using the same SSL certificate I have for >> shakerweather.com that is forwarding to port 80 to serve the page. >> >> Cloudflare: >> TYPE NAME CONTENT >> CNAME mqtt shakerweather.com >> A shakerweather.com static wan IP >> >> *weewx.conf* >> [[MQTT]] >> server_url = mqtt://user:[email protected]:1883/ >> <http://user:[email protected]:1883/> >> >> topic = weather >> unit_system = US >> binding = archive, loop >> aggregation = aggregate >> >> *skin.conf* >> # MQTT Websockets defaults >> mqtt_websockets_enabled = 1 >> mqtt_websockets_host = "mqtt.shakerweather.com" >> mqtt_websockets_port = 443 >> >> mqtt_websockets_ssl = 1 >> mqtt_websockets_topic = "weather/loop" >> disconnect_live_website_visitor = 1800000 >> >> *mosquitto.conf* >> persistence false >> >> # mqtt >> listener 1883 >> protocol mqtt >> >> # websockets >> listener 9001 >> protocol websockets >> >> >> allow_anonymous true >> password_file /etc/mosquitto/passwd >> >> acl_file /etc/mosquitto/acl >> On Saturday, July 24, 2021 at 8:33:43 AM UTC-4 [email protected] >> wrote: >> >>> I had a number of struggles with this when I setup the Belchertown skin >>> on my Raspberry Pi hosting my weather site, www.largoweather.com. I >>> think the issue is that you need your mqtt_websockets_port to be set to 443 >>> as the websocket traffic is getting filtered out by your firewall >>> (9001->8083) >>> >>> Here is what I did to get this to work on largoweather.com: >>> >>> 1. Setup Cloudflare to manage the DNS proxy on 2 domains: >>> - largoweather.com (A Record) >>> - wx.largoweather.com (aka your mqtt.beldenserver.com) - CNAME >>> Record pointing the content to largoweather.com >>> - Setup my SSL/TLS to Strict. I am using Cloudflare to offload my SSL >>> processing so essentially all traffic is coming in through 443. >>> >>> These are pointed to my public IP address which is dynamic. I use a >>> shell script to update cloudflare' content to keep my public IP current for >>> their system >>> >>> 2. NGNIX Proxy Manager : I use this program to manage my NGINX instance >>> that acts as a reverse proxy manager for my domains: >>> 2.1 : Setup: >>> - largoweather.com : Setup to point to my local server ip >>> address. The scheme is http. I use the program's lets encrypt function to >>> get a SSL certificate and force SSL traffic to my final weewx website. >>> - wx.largoweather.com : This handles my sockets setup. I >>> forward all traffic to port 9001 and use the same ssl certificate issued >>> for largoweather. I force SSL on this setup as well. >>> >>> 3. mosquitto configuration : I setup mosquitto on the same server as my >>> weewx install since everything is running on the pi. here is my >>> mosquitto.conf: >>> pid_file /var/run/mosquitto.pid >>> persistence true >>> persistence_location /var/lib/mosquitto/ >>> log_type error >>> websockets_log_level 1023 >>> connection_messages true >>> log_dest file /mnt/*****/weewx/logs/mosquitto.log >>> >>> allow_anonymous true >>> password_file /etc/mosquitto/passwd >>> acl_file /etc/mosquitto/acl >>> >>> listener 9001 >>> protocol websockets >>> >>> listener 1883 >>> protocol mqtt >>> log_type error >>> >>> >>> 4. weewx/Belchertown configuration: Here I setup my MQTT to talk to my >>> local IP address on port 1883. Remember the traffic is all coming in on >>> port 443, so that is the port I need Belchertown skin to essentially >>> connect to resolve the web sockets requests. >>> >>> [[MQTT]] >>> server_url = mqtt://joeuser:xxxxxxx@<YOUR-LOCAL-SERVER-IP>:1883/ >>> topic = weather >>> unit_system = US >>> binding = archive, loop >>> aggregation = aggregate >>> [[Belchertown]] >>> skin = Belchertown >>> enable = True >>> >>> [[[Extras]]] >>> site_title = Largo Weather >>> mqtt_websockets_enabled = 1 >>> mqtt_websockets_host = wxsocket.largoweather.com >>> mqtt_websockets_port = 443 >>> mqtt_websockets_topic = weather/loop >>> mqtt_websockets_ssl = 1 >>> disconnect_live_website_visitor = 1800000 >>> >>> I hope this helps! >>> >>> Doug Jenkins >>> >>> >>> On Saturday, July 24, 2021 at 8:07:23 AM UTC-4 [email protected] >>> wrote: >>> >>>> Thanks Les. I think you have helped clear some things up. For the sake >>>> of clarity and getting to the end state I desire and your suggestion let's >>>> focus on locally hosting (my first setup) over the cloud broker. I have a >>>> robust homelab and really like to keep things in house. >>>> >>>> *think the first configuration, with the local mqtt broker isn’t going >>>> to work because mqtt_websockets_host is set to localhost, which will only >>>> resolve to the weewx/mqtt server when the web browser is running on that >>>> server. You need to set something here that will resolve to the >>>> weewx/mqtt >>>> server from any client that you want to get realtime updates. * >>>> >>>> This makes sense. I initially was on this path and changed the >>>> mqtt_websockets host to the ip address of the weewx/mqtt server. This did >>>> not work for other clients on the LAN which seemed strange given how I >>>> have >>>> lots of locally running services on a few different servers in my home lab >>>> and I access them all via device IP and port, with a few that are >>>> accessible externally via reverse proxy. >>>> >>>> >>>> *You need a DNS name that will resolve to your firewall and get >>>> port-forwarded (for port 8083) to the weewx/mqtt server, >>>> for mqtt_websockets_host. That should enable external access. * >>>> >>>> I can use my mqtt.beldenserver.com DNS name I have setup at cloudflare >>>> for this. I am running OPNSense for my firewall so I should be able to do >>>> anything. Right now I have a firewall rule setup to send port 80/443 >>>> traffic to my NGINX Reverse Proxy where I have several DNS addresses >>>> pointing to different services on different servers. One of these takes >>>> shakerweather.com through 80/443 and through the reverse proxy and >>>> points to the weewx/mqtt server to return the webpage (which is working >>>> fine) >>>> >>>> So you are saying I just need to add a rule to forward external port >>>> 8083 requests to the weewx/mqtt server IP and port? >>>> >>>> *And, if your firewall will do hairpinning, it should work internally >>>> as well. It may take some magic with forwarding/masquerading rules on the >>>> firewall to get hairpinning to work. (The alternative for internal access >>>> is to have an internal DNS server that resolves that hostname directly to >>>> the internal IP of the weewx/mqtt server for clients on the internal >>>> network.)* >>>> >>>> My OPNSense firewall should be able to do hairpinning. I read on that >>>> briefly as I have only heard of the term and not too familiar with it. >>>> >>>> >>>> >>>> >>>> On Saturday, July 24, 2021 at 2:24:32 AM UTC-4 ln77 wrote: >>>> >>>>> I think the first configuration, with the local mqtt broker isn’t >>>>> going to work because mqtt_websockets_host is set to localhost, which >>>>> will >>>>> only resolve to the weewx/mqtt server when the web browser is running on >>>>> that server. You need to set something here that will resolve to the >>>>> weewx/mqtt server from any client that you want to get realtime updates. >>>>> >>>>> Not sure why the second config, with the cloud mqtt broker, isn’t >>>>> working. Are you sure the mqtt broker is configured for SSL on port 8883? >>>>> You might put “log_success = true” in the weewx [[MQTT]] config and see >>>>> if >>>>> the log messages tell you anything useful. >>>>> >>>>> Or forget about the cloud server and go back to getting the first >>>>> config working. You need a DNS name that will resolve to your firewall >>>>> and >>>>> get port-forwarded (for port 8083) to the weewx/mqtt server, >>>>> for mqtt_websockets_host. That should enable external access. And, if >>>>> your >>>>> firewall will do hairpinning, it should work internally as well. It may >>>>> take some magic with forwarding/masquerading rules on the firewall to get >>>>> hairpinning to work. (The alternative for internal access is to have an >>>>> internal DNS server that resolves that hostname directly to the internal >>>>> IP >>>>> of the weewx/mqtt server for clients on the internal network.) >>>>> >>>>> -Les >>>>> >>>>> >>>>> >>>>> On 23 Jul 2021, at 21:05, Matt Johnson <[email protected]> wrote: >>>>> >>>>> I've been trouble shooting getting the Belchertown skin MQTT Websocket >>>>> real time updates to work on my site shakerweather.com for a lot of >>>>> this week. >>>>> >>>>> I have WeeWx installed on a dedicated thin client on Ubuntu 20.04 LTS >>>>> and have Mosquitto and NGINX installed on the same machine. Running bare >>>>> metal, no docker or VMs here. >>>>> >>>>> External access to WeeWx website is handled via NGINX reverse proxy >>>>> manager with SSL certs on a different server via docker. Requests to >>>>> shakerweather.com are sent to the proxy server and then to the WeeWx >>>>> machine. >>>>> >>>>> With this setup, the site is served up fine internally and externally >>>>> with the updates at archive intervals every 5 minutes. >>>>> >>>>> I know that the weewx-mqtt extension is installed correctly as I have >>>>> been able to test it locally and get the websocket updates to work >>>>> perfectly with the following configs: >>>>> >>>>> *weewx.conf* >>>>> [[MQTT]] >>>>> server_url = mqtt://user:pw@localhost:8883/ >>>>> topic = weather >>>>> unit_system = US >>>>> binding = archive, loop >>>>> aggregation = aggregate >>>>> >>>>> *skin.conf* >>>>> # MQTT Websockets defaults >>>>> mqtt_websockets_enabled = 1 >>>>> mqtt_websockets_host = "localhost" >>>>> mqtt_websockets_port = 8083 >>>>> mqtt_websockets_ssl = 0 >>>>> mqtt_websockets_topic = "weather/loop" >>>>> disconnect_live_website_visitor = 1800000 >>>>> >>>>> I am only able to see the the real time updates on the local machine >>>>> only with WeeWx and Mosquitto. If I try to access it by IP address >>>>> elsewhere on my LAN on other clients it does not connect and eventually >>>>> fails. No luck externally either - despite my NGINX Reverse Proxy Manger >>>>> handling serving the page and SSL certs the websocket real time updates >>>>> don't pass through. That was my original thought of how it would work. >>>>> >>>>> After much trial and error, and reading every thread imaginable on >>>>> this along with many messages and some correspondence with Pat O'Brien I >>>>> decided to go ahead and setup a Digital Ocean Ubuntu VM and install >>>>> Mosquito there to serve as a cloud broker. I followed Pat's instructions >>>>> exactly as he outlines in setting up the cloud broker: >>>>> https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/ >>>>> >>>>> I have the cloud MQTT broker installed correctly at Digital Ocean with >>>>> Let's Encrypt, and ran tests on it. Messages can be sent when >>>>> authenticated, ports are open, etc. However, I can get no further with >>>>> the >>>>> websockets real time updates than "Connected. Waiting for data". If I >>>>> reboot the cloud MQTT broker I immediately get a disconnected message on >>>>> the website so it does appear to be connecting and waiting for data. >>>>> Somehow the data is simply not transferring from my WeeWx client to the >>>>> cloud MQTT broker at Digital Ocean. The other weird thing is if I try to >>>>> access shakerweather.com or the website by local IP address on the >>>>> machine that hosts WeeWx I always get a failed message, won't even >>>>> connect >>>>> to the server. However, any other client on my LAN and external on WAN >>>>> does >>>>> not have this issue. >>>>> >>>>> Here are my current configs: >>>>> >>>>> *weewx.conf* >>>>> [[MQTT]] >>>>> server_url = mqtt://user:[email protected]:8883/ >>>>> topic = weather >>>>> unit_system = US >>>>> binding = archive, loop >>>>> aggregation = aggregate >>>>> [[[tls]]] >>>>> tls_version = tlsv1 >>>>> ca_certs = /etc/ssl/certs/ca-certificates.crt >>>>> >>>>> *skin.conf* >>>>> # MQTT Websockets defaults >>>>> mqtt_websockets_enabled = 1 >>>>> mqtt_websockets_host = "mqtt.beldenserver.com" >>>>> mqtt_websockets_port = 8083 >>>>> mqtt_websockets_ssl = 1 >>>>> mqtt_websockets_topic = "weather/loop" >>>>> disconnect_live_website_visitor = 1800000 >>>>> >>>>> At this point, I have spent 20+ hours on this and hoping someone here >>>>> can point me in the right direction, it seems data is just not feeding >>>>> the >>>>> MQTT topic. I'm fine with using Digital Ocean as a cloud MQTT server just >>>>> to get it up and running. My preferred state is eventually to selfhost it >>>>> all. >>>>> >>>>> Thanks in advance for anything I may be overlooking, advice or >>>>> possible solutions. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/38183ff6-d5d5-4151-a1b3-93fd618aef5cn%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/38183ff6-d5d5-4151-a1b3-93fd618aef5cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/abff06e6-95ad-490e-b9f4-4c86a87fcf4cn%40googlegroups.com.
