Doing more trouble shooting. I have narrowed it down to a SSL issue.
To confirm this I disabled SSL at cloudflare and disabled force SSL for
shakerweather.com with my NGINX reverse proxy server and ran with this
setup:
*weewx.config*
[[MQTT]]
server_url = mqtt://usr:[email protected]:1883/
topic = weather
unit_system = US
binding = archive, loop
aggregation = aggregate
*skin.conf*
# MQTT Websockets defaults
mqtt_websockets_enabled = 1
mqtt_websockets_host = "mqtt.shakerweather.com"
mqtt_websockets_port = 9001
mqtt_websockets_ssl = 0
mqtt_websockets_topic = "weather/loop"
disconnect_live_website_visitor = 1800000
*mosquitto.conf*
persistence false
# mqtt
listener 1883
protocol mqtt
# websockets
listener 9001
protocol websockets
allow_anonymous true
password_file /etc/mosquitto/passwd
acl_file /etc/mosquitto/acl
I am able to see the real time websocket updates internally and externally
at http://shakerweather.com (I'll leave it up for a short time so if it
doesn't work if you click on it that could be why)
So at least I have narrowed down the issue. Still not sure how to resolve
to ensure all is secured with SSL....
On Saturday, July 24, 2021 at 10:23:03 AM UTC-4 Matt Johnson wrote:
> Your setup seems similar to mine so I went for it. Still not working.
>
> I went ahead and added a mqtt CNAME record to the shakerweather.com
> cloudflare setup. I also added mqtt.shakerweather.com to my NGINX Proxy
> Server to forward to port 9001 using the same SSL certificate I have for
> shakerweather.com that is forwarding to port 80 to serve the page.
>
> Cloudflare:
> TYPE NAME CONTENT
> CNAME mqtt shakerweather.com
> A shakerweather.com static wan IP
>
> *weewx.conf*
> [[MQTT]]
> server_url = mqtt://user:[email protected]:1883/
> <http://user:[email protected]:1883/>
>
> topic = weather
> unit_system = US
> binding = archive, loop
> aggregation = aggregate
>
> *skin.conf*
> # MQTT Websockets defaults
> mqtt_websockets_enabled = 1
> mqtt_websockets_host = "mqtt.shakerweather.com"
> mqtt_websockets_port = 443
>
> mqtt_websockets_ssl = 1
> mqtt_websockets_topic = "weather/loop"
> disconnect_live_website_visitor = 1800000
>
> *mosquitto.conf*
> persistence false
>
> # mqtt
> listener 1883
> protocol mqtt
>
> # websockets
> listener 9001
> protocol websockets
>
>
> allow_anonymous true
> password_file /etc/mosquitto/passwd
>
> acl_file /etc/mosquitto/acl
> On Saturday, July 24, 2021 at 8:33:43 AM UTC-4 [email protected]
> wrote:
>
>> I had a number of struggles with this when I setup the Belchertown skin
>> on my Raspberry Pi hosting my weather site, www.largoweather.com. I
>> think the issue is that you need your mqtt_websockets_port to be set to 443
>> as the websocket traffic is getting filtered out by your firewall
>> (9001->8083)
>>
>> Here is what I did to get this to work on largoweather.com:
>>
>> 1. Setup Cloudflare to manage the DNS proxy on 2 domains:
>> - largoweather.com (A Record)
>> - wx.largoweather.com (aka your mqtt.beldenserver.com) - CNAME Record
>> pointing the content to largoweather.com
>> - Setup my SSL/TLS to Strict. I am using Cloudflare to offload my SSL
>> processing so essentially all traffic is coming in through 443.
>>
>> These are pointed to my public IP address which is dynamic. I use a
>> shell script to update cloudflare' content to keep my public IP current for
>> their system
>>
>> 2. NGNIX Proxy Manager : I use this program to manage my NGINX instance
>> that acts as a reverse proxy manager for my domains:
>> 2.1 : Setup:
>> - largoweather.com : Setup to point to my local server ip
>> address. The scheme is http. I use the program's lets encrypt function to
>> get a SSL certificate and force SSL traffic to my final weewx website.
>> - wx.largoweather.com : This handles my sockets setup. I
>> forward all traffic to port 9001 and use the same ssl certificate issued
>> for largoweather. I force SSL on this setup as well.
>>
>> 3. mosquitto configuration : I setup mosquitto on the same server as my
>> weewx install since everything is running on the pi. here is my
>> mosquitto.conf:
>> pid_file /var/run/mosquitto.pid
>> persistence true
>> persistence_location /var/lib/mosquitto/
>> log_type error
>> websockets_log_level 1023
>> connection_messages true
>> log_dest file /mnt/*****/weewx/logs/mosquitto.log
>>
>> allow_anonymous true
>> password_file /etc/mosquitto/passwd
>> acl_file /etc/mosquitto/acl
>>
>> listener 9001
>> protocol websockets
>>
>> listener 1883
>> protocol mqtt
>> log_type error
>>
>>
>> 4. weewx/Belchertown configuration: Here I setup my MQTT to talk to my
>> local IP address on port 1883. Remember the traffic is all coming in on
>> port 443, so that is the port I need Belchertown skin to essentially
>> connect to resolve the web sockets requests.
>>
>> [[MQTT]]
>> server_url = mqtt://joeuser:xxxxxxx@<YOUR-LOCAL-SERVER-IP>:1883/
>> topic = weather
>> unit_system = US
>> binding = archive, loop
>> aggregation = aggregate
>> [[Belchertown]]
>> skin = Belchertown
>> enable = True
>>
>> [[[Extras]]]
>> site_title = Largo Weather
>> mqtt_websockets_enabled = 1
>> mqtt_websockets_host = wxsocket.largoweather.com
>> mqtt_websockets_port = 443
>> mqtt_websockets_topic = weather/loop
>> mqtt_websockets_ssl = 1
>> disconnect_live_website_visitor = 1800000
>>
>> I hope this helps!
>>
>> Doug Jenkins
>>
>>
>> On Saturday, July 24, 2021 at 8:07:23 AM UTC-4 [email protected] wrote:
>>
>>> Thanks Les. I think you have helped clear some things up. For the sake
>>> of clarity and getting to the end state I desire and your suggestion let's
>>> focus on locally hosting (my first setup) over the cloud broker. I have a
>>> robust homelab and really like to keep things in house.
>>>
>>> *think the first configuration, with the local mqtt broker isn’t going
>>> to work because mqtt_websockets_host is set to localhost, which will only
>>> resolve to the weewx/mqtt server when the web browser is running on that
>>> server. You need to set something here that will resolve to the weewx/mqtt
>>> server from any client that you want to get realtime updates. *
>>>
>>> This makes sense. I initially was on this path and changed the
>>> mqtt_websockets host to the ip address of the weewx/mqtt server. This did
>>> not work for other clients on the LAN which seemed strange given how I have
>>> lots of locally running services on a few different servers in my home lab
>>> and I access them all via device IP and port, with a few that are
>>> accessible externally via reverse proxy.
>>>
>>>
>>> *You need a DNS name that will resolve to your firewall and get
>>> port-forwarded (for port 8083) to the weewx/mqtt server,
>>> for mqtt_websockets_host. That should enable external access. *
>>>
>>> I can use my mqtt.beldenserver.com DNS name I have setup at cloudflare
>>> for this. I am running OPNSense for my firewall so I should be able to do
>>> anything. Right now I have a firewall rule setup to send port 80/443
>>> traffic to my NGINX Reverse Proxy where I have several DNS addresses
>>> pointing to different services on different servers. One of these takes
>>> shakerweather.com through 80/443 and through the reverse proxy and
>>> points to the weewx/mqtt server to return the webpage (which is working
>>> fine)
>>>
>>> So you are saying I just need to add a rule to forward external port
>>> 8083 requests to the weewx/mqtt server IP and port?
>>>
>>> *And, if your firewall will do hairpinning, it should work internally as
>>> well. It may take some magic with forwarding/masquerading rules on the
>>> firewall to get hairpinning to work. (The alternative for internal access
>>> is to have an internal DNS server that resolves that hostname directly to
>>> the internal IP of the weewx/mqtt server for clients on the internal
>>> network.)*
>>>
>>> My OPNSense firewall should be able to do hairpinning. I read on that
>>> briefly as I have only heard of the term and not too familiar with it.
>>>
>>>
>>>
>>>
>>> On Saturday, July 24, 2021 at 2:24:32 AM UTC-4 ln77 wrote:
>>>
>>>> I think the first configuration, with the local mqtt broker isn’t
>>>> going to work because mqtt_websockets_host is set to localhost, which will
>>>> only resolve to the weewx/mqtt server when the web browser is running on
>>>> that server. You need to set something here that will resolve to the
>>>> weewx/mqtt server from any client that you want to get realtime updates.
>>>>
>>>> Not sure why the second config, with the cloud mqtt broker, isn’t
>>>> working. Are you sure the mqtt broker is configured for SSL on port 8883?
>>>> You might put “log_success = true” in the weewx [[MQTT]] config and see
>>>> if
>>>> the log messages tell you anything useful.
>>>>
>>>> Or forget about the cloud server and go back to getting the first
>>>> config working. You need a DNS name that will resolve to your firewall
>>>> and
>>>> get port-forwarded (for port 8083) to the weewx/mqtt server,
>>>> for mqtt_websockets_host. That should enable external access. And, if
>>>> your
>>>> firewall will do hairpinning, it should work internally as well. It may
>>>> take some magic with forwarding/masquerading rules on the firewall to get
>>>> hairpinning to work. (The alternative for internal access is to have an
>>>> internal DNS server that resolves that hostname directly to the internal
>>>> IP
>>>> of the weewx/mqtt server for clients on the internal network.)
>>>>
>>>> -Les
>>>>
>>>>
>>>>
>>>> On 23 Jul 2021, at 21:05, Matt Johnson <[email protected]> wrote:
>>>>
>>>> I've been trouble shooting getting the Belchertown skin MQTT Websocket
>>>> real time updates to work on my site shakerweather.com for a lot of
>>>> this week.
>>>>
>>>> I have WeeWx installed on a dedicated thin client on Ubuntu 20.04 LTS
>>>> and have Mosquitto and NGINX installed on the same machine. Running bare
>>>> metal, no docker or VMs here.
>>>>
>>>> External access to WeeWx website is handled via NGINX reverse proxy
>>>> manager with SSL certs on a different server via docker. Requests to
>>>> shakerweather.com are sent to the proxy server and then to the WeeWx
>>>> machine.
>>>>
>>>> With this setup, the site is served up fine internally and externally
>>>> with the updates at archive intervals every 5 minutes.
>>>>
>>>> I know that the weewx-mqtt extension is installed correctly as I have
>>>> been able to test it locally and get the websocket updates to work
>>>> perfectly with the following configs:
>>>>
>>>> *weewx.conf*
>>>> [[MQTT]]
>>>> server_url = mqtt://user:pw@localhost:8883/
>>>> topic = weather
>>>> unit_system = US
>>>> binding = archive, loop
>>>> aggregation = aggregate
>>>>
>>>> *skin.conf*
>>>> # MQTT Websockets defaults
>>>> mqtt_websockets_enabled = 1
>>>> mqtt_websockets_host = "localhost"
>>>> mqtt_websockets_port = 8083
>>>> mqtt_websockets_ssl = 0
>>>> mqtt_websockets_topic = "weather/loop"
>>>> disconnect_live_website_visitor = 1800000
>>>>
>>>> I am only able to see the the real time updates on the local machine
>>>> only with WeeWx and Mosquitto. If I try to access it by IP address
>>>> elsewhere on my LAN on other clients it does not connect and eventually
>>>> fails. No luck externally either - despite my NGINX Reverse Proxy Manger
>>>> handling serving the page and SSL certs the websocket real time updates
>>>> don't pass through. That was my original thought of how it would work.
>>>>
>>>> After much trial and error, and reading every thread imaginable on this
>>>> along with many messages and some correspondence with Pat O'Brien I
>>>> decided
>>>> to go ahead and setup a Digital Ocean Ubuntu VM and install Mosquito there
>>>> to serve as a cloud broker. I followed Pat's instructions exactly as he
>>>> outlines in setting up the cloud broker:
>>>> https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/
>>>>
>>>> I have the cloud MQTT broker installed correctly at Digital Ocean with
>>>> Let's Encrypt, and ran tests on it. Messages can be sent when
>>>> authenticated, ports are open, etc. However, I can get no further with the
>>>> websockets real time updates than "Connected. Waiting for data". If I
>>>> reboot the cloud MQTT broker I immediately get a disconnected message on
>>>> the website so it does appear to be connecting and waiting for data.
>>>> Somehow the data is simply not transferring from my WeeWx client to the
>>>> cloud MQTT broker at Digital Ocean. The other weird thing is if I try to
>>>> access shakerweather.com or the website by local IP address on the
>>>> machine that hosts WeeWx I always get a failed message, won't even connect
>>>> to the server. However, any other client on my LAN and external on WAN
>>>> does
>>>> not have this issue.
>>>>
>>>> Here are my current configs:
>>>>
>>>> *weewx.conf*
>>>> [[MQTT]]
>>>> server_url = mqtt://user:[email protected]:8883/
>>>> topic = weather
>>>> unit_system = US
>>>> binding = archive, loop
>>>> aggregation = aggregate
>>>> [[[tls]]]
>>>> tls_version = tlsv1
>>>> ca_certs = /etc/ssl/certs/ca-certificates.crt
>>>>
>>>> *skin.conf*
>>>> # MQTT Websockets defaults
>>>> mqtt_websockets_enabled = 1
>>>> mqtt_websockets_host = "mqtt.beldenserver.com"
>>>> mqtt_websockets_port = 8083
>>>> mqtt_websockets_ssl = 1
>>>> mqtt_websockets_topic = "weather/loop"
>>>> disconnect_live_website_visitor = 1800000
>>>>
>>>> At this point, I have spent 20+ hours on this and hoping someone here
>>>> can point me in the right direction, it seems data is just not feeding the
>>>> MQTT topic. I'm fine with using Digital Ocean as a cloud MQTT server just
>>>> to get it up and running. My preferred state is eventually to selfhost it
>>>> all.
>>>>
>>>> Thanks in advance for anything I may be overlooking, advice or possible
>>>> solutions.
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "weewx-user" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/weewx-user/38183ff6-d5d5-4151-a1b3-93fd618aef5cn%40googlegroups.com
>>>>
>>>> <https://groups.google.com/d/msgid/weewx-user/38183ff6-d5d5-4151-a1b3-93fd618aef5cn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>>
>>>>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/507ba8a1-b0e8-4ea0-9037-0d6e7beda841n%40googlegroups.com.
