*What is emitting mqtt ? *
rtl_433 runs as a service on the same system as weewx is running on, set to
output MQTT topics in SI units, via rtl_433 -C si -F mqtt.
*Are there actual duplicate mqtt messages with the same timestamps ?*
Yes and no. The majority of all of my sensors output multiple packets, for
redundancy. Many- but not all- have a topic that is something to the
effect of "sequence", but but don't otherwise differ in the payloads of the
subscribed topics (notwithstanding Acurite's use of multiple message types
with mixed topics across them). Here's a message type 39 with wind avg,
uv, lux, and lightning topics included. The packet was broadcast in
triplicate, three copies broadcast and received, with sequence_num 1, 2,
and 3:
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/time 2025-04-01 18:50:36
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/id 571
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/channel A
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/sequence_num 0
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/battery_ok 1
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/message_type 39
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_avg_km_h 3.21869
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/uv 0
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/lux 5910
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 344
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 22
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/exception 0
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/raw_msg c23be7810084cf5696a4
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/mod ASK
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/time 2025-04-01 18:50:36
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/id 571
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/channel A
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/sequence_num 1
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/battery_ok 1
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/message_type 39
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_avg_km_h 3.21869
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/uv 0
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/lux 5910
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 344
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 22
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/exception 0
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/raw_msg c63be7810084cf5696a8
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/mod ASK
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/time 2025-04-01 18:50:36
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/id 571
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/channel A
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/sequence_num 2
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/battery_ok 1
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/message_type 39
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_avg_km_h 3.21869
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/uv 0
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/lux 5910
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 344
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 22
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/exception 0
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/raw_msg ca3be7810084cf5696ac
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/mod ASK
*How is your mqtt subscribe configured ?*
What I pasted above was pulled straight from "mosquitto_sub -v -h localhost
-t "rtl_433/pi4b8/devices/Acurite-Atlas/A/571/#"". The weewx.conf section
looks like this:
[MQTTSubscribeDriver]
driver = user.MQTTSubscribe
host = 10.19.76.14
port = 1883
keepalive = 20
username = None
password = None
[[message_callback]]
type = individual
[[topics]]
unit_system = METRICWX
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/humidity]]]
name = outHumidity
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/rain_mm]]]
name = rain
contains_total = true
[[[br-weewx/Acurite-Atlas/A/571/rain_mm]]]
**this one comes from a relay broker so I can capture the same topic
twice and handle it differently for QC purposes**
name = snow
contains_total = false
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/temperature_C]]]
name = outTemp
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_avg_km_h]]]
name = windSpeed
units = km_per_hour
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/wind_dir_deg]]]
name = windDir
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/battery_ok]]]
name = outTempBatteryStatus
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/lux]]]
name = luminosity
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/uv]]]
name = UV
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count]]]
name = lightning_strike_count
contains_total = true
[[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance]]]
name = lightning_distance
units = km
*Which mqtt subscribe variant and version are you using ?*
pi@pi4b8:/etc/systemd/system $ weectl extension list
Using configuration file /etc/weewx/weewx.conf
Extension Name Version Description
MQTTSubscribe 2.3.0 Augment WeeWX records or packets with MQTT data.
*What do a few received mqtt messages look like ?*
Native mosquitto_sub gets what I pasted above. MQTTSubscribe doesn't put
(pretty much) anything in the log unless debug is enabled.
*What do a few weewx loop messages with debug=1 look like ?*
with debug enabled, the weewx log looks something like this (filtered to
just lightning topics):
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> incoming
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count
'lightning_strike_count' '0.0'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> incoming
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count
'lightning_strike_count' '0.0'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> incoming
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count
'lightning_strike_count' '0.0'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> incoming
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance
'lightning_distance' '22.0'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> incoming
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance
'lightning_distance' '22.0'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> incoming
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance
'lightning_distance' '22.0'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> outgoing
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 'dateTime'
'1743544003.5789976', 'lightning_strike_count' '0.0', 'usUnits' '17'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> outgoing
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 'dateTime'
'1743544003.5918882', 'lightning_strike_count' '0.0', 'usUnits' '17'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> outgoing
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 'dateTime'
'1743544003.5977087', 'lightning_strike_count' '0.0', 'usUnits' '17'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> outgoing
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 'dateTime'
'1743544003.5803194', 'lightning_distance' '22.0', 'usUnits' '17'
4/1/25 17:46:43 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> outgoing
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 'dateTime'
'1743544003.5927773', 'lightning_distance' '22.0', 'usUnits' '17'
4/1/25 17:46:44 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
data-> final loop packet is
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174643
EDT (1743544003) 'dateTime' '1743544003.599552', 'lightning_distance'
'22.0', 'usUnits' '17'
4/1/25 17:46:44 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
TopicManager data-> outgoing
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 'dateTime'
'1743544003.599552', 'lightning_distance' '22.0', 'usUnits' '17'
4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
data-> final loop packet is
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654
EDT (1743544014) 'dateTime' '1743544014.06267', 'lightning_strike_count'
'0.0', 'usUnits' '17'
4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
data-> final loop packet is
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654
EDT (1743544014) 'dateTime' '1743544014.066883', 'lightning_strike_count'
'0.0', 'usUnits' '17'
4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
data-> final loop packet is
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count 2025-04-01 174654
EDT (1743544014) 'dateTime' '1743544014.0723326', 'lightning_strike_count'
'0.0', 'usUnits' '17'
4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
data-> final loop packet is
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654
EDT (1743544014) 'dateTime' '1743544014.063789', 'lightning_distance'
'22.0', 'usUnits' '17'
4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
data-> final loop packet is
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654
EDT (1743544014) 'dateTime' '1743544014.0683196', 'lightning_distance'
'22.0', 'usUnits' '17'
4/1/25 17:46:54 pi4b8 weewxd[1166914]: DEBUG user.MQTTSubscribe (Driver)
data-> final loop packet is
rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance 2025-04-01 174654
EDT (1743544014) 'dateTime' '1743544014.0791476', 'lightning_distance'
'22.0', 'usUnits' '17'
*What do you see ?*
MQTTSubscribe looks to be passing the topic payload as the observation.
This is expected, as there's no logic applied to a non-cumulative value.
*What do you expect/hope to see ?*
StdCalibrate should be applying the correction formula to loop packets, or
at least that's my understanding. All of my WX sensors are configured
through MQTT. When weewx receives an observation datapoint, regardless of
the driver producing it, it should apply the correction before any record
is made. In this case,
"lightning_distance = lightning_distance if lightning_strike_count > 0 else
None" should be applied akin to =IF(lightning_strike_count > 0,
lightning_distance, "None"), but instead of evaluating to the False
condition, it's evaluating to True.
On Tue, Apr 1, 2025 at 6:14 PM vince <vinceska...@gmail.com> wrote:
> Andrew - I think you need to more clearly state your setup and which
> problem(s) you are asking for help with because I certainly got very lost
> in your posts…..
>
> What is emitting mqtt ?
> Are there actual duplicate mqtt messages with the same timestamps ?
> How is your mqtt subscribe configured ?
> Which mqtt subscribe variant and version are you using ?
>
> What do a few received mqtt messages look like ?
> What do a few weewx loop messages with debug=1 look like ?
> What do you see ?
> What do you expect/hope to see ?
>
> On Tuesday, April 1, 2025 at 1:43:56 PM UTC-7 gjr80 wrote:
>
>> A few comments.
>>
>> I don't see multiple copies of a loop packet arriving causing a problem.
>> Archive record obs that are averaged (eg outTemp) will be the same
>> irrespective of how many time the loop packet is 'accumulated'. Per-period
>> obs such as rain and, in this case, lightning_strike_count will
>> similarly be unaffected as the difference for each such obs between
>> successive duplicate loop packets will be zero. Of course cumulative obs
>> will be nonsense, but as WeeWX does not use cumulative obs that should not
>> be a problem.
>>
>> Also, keep in mind how the [StdCalibrate] [[Corrections]] operate. Each
>> [StdCalibrate]
>> [[Corrections]] option value is used as the argument to a python eval()
>> statement; if the eval() statement causes a python exception to be
>> raised the correction is silently ignored (and if it exists the destination
>> obs is left unchanged, if it does not exist it is not created). I would
>> suggest your main issue is your lightning_distance correction not
>> working as intended (are you sure lightning_strike_count is indeed 0 or
>> is it None?). Also, the 2nd of your three lightning_distance corrections
>> is the one you use; WeeWX does not use 'nan' and use of Null will raise
>> a python exception.
>>
>> I would suggest you set some appropriate debug settings (I am not
>> familiar with the MQTTSubscribe driver to know what debug settings are
>> required) in place such that loop packets emitted by the MQTTSubscribe driver
>> are logged. Then you will know exactly what is being fed to WeeWX and how
>> to ensure it is processed/recorded correctly.
>>
>> Gary
>> On Wednesday, 2 April 2025 at 04:52:32 UTC+10 Andrew McGinnis wrote:
>>
>>> ETA- 5000+ strike count/min comes from the sensor broadcasting packets
>>> in triplicate, and every 10 seconds for lightning. So up to 18 messages in
>>> each 1-min interval, with what was the true cumulative count of 303,
>>> resulted in 5454 strikes "recorded".
>>>
>>> If anyone has a way to suppress duplicate MQTT messages, I'm all ears.
>>> I get why sensors will send a packet multiple times to maximize the chances
>>> of reception at a station, but weewx (or, at least MQTTSubscribeDriver)
>>> sees each one and records it as a separate datapoint to aggregate in the
>>> archive period. Mostly harmless, but not entirely accurate.
>>> On Tuesday, April 1, 2025 at 2:28:50 PM UTC-4 Andrew McGinnis wrote:
>>>
>>>> I just set up an Acurite Atlas with the lightning detector sensor
>>>> add-on. The strike_count that it outputs is cumulative (after I found I
>>>> had 5000+ lightning strikes per 1min archive interval). I have it, and the
>>>> strike_distance outputs mapped to weewx as below. rtl_433 is converting
>>>> strike_distance from the imperial 'miles' that is natively output, to
>>>> metric 'km', hence the additional units switch:
>>>>
>>>> [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_count]]]
>>>> name = lightning_strike_count
>>>> contains_total = true
>>>> [[[rtl_433/pi4b8/devices/Acurite-Atlas/A/571/strike_distance]]]
>>>> name = lightning_distance
>>>> units = km
>>>>
>>>> The sensor output always has a strike count and distance, but since the
>>>> strike count is cumulative, there should be no recorded distance if the
>>>> delta is 0.
>>>>
>>>> In StdCalibrate/Corrections, I have these:
>>>> radiation = luminosity / 126.7 if luminosity is not None else None
>>>> (<----this is working as expected)
>>>> lightning_distance = lightning_distance if lightning_strike_count > 0
>>>> else float('nan') (<----does not work)
>>>> lightning_distance = lightning_distance if lightning_strike_count > 0
>>>> else None (<----does not work)
>>>> lightning_distance = lightning_distance if lightning_strike_count > 0
>>>> else Null (<----does not work)
>>>>
>>>> The *radiation *observation is being recorded as 0 when *luminosity *is
>>>> recorded as 0. But, since 0 is a valid possible output, if 0 != None, then
>>>> 0/126.7 = 0. Fair enough.
>>>> The *lightning_distance* should be evaluating as 0 or None, but
>>>> instead is being recorded as the sensor's output value, while
>>>> *lightning_strike_count* is recorded as 0. I went into the db and ran
>>>> the following, then rebuilt the affected dailies:
>>>>
>>>> - update archive set lightning_strike_count = 0 where
>>>> lightning_strike_count > 5;
>>>> - update archive set lightning_distance = 0 where
>>>> lightning_strike_count = 0;
>>>>
>>>> strike_count > 5 was because I caught my contains_total omission mid
>>>> day yesterday, and there was thunder last night. Strikes before the storm
>>>> were largely invalid, but distance wasn't necessarily.
>>>>
>>>> [image: Screenshot 2025-04-01 141609.png]
>>>> Straight away, *lightning_distance* is being recorded as the sensor's
>>>> value, instead of 0, or ideally None/null.
>>>> 0 is a valid possible distance, assuming I get a direct strike. Without
>>>> a strike, it's not 0 distance- there *isn't* a distance.
>>>>
>>>> Rather than continue trying random changes to the Corrections logic,
>>>> can anyone point to where the issue or mistake may lie? Is it something to
>>>> do with the referenced *lightning_strike_count* being a cumulative
>>>> number? Do I just need to yield and make the logic evaluate to 0?
>>>>
>>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-user/b3O1IzVjiCc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> To view this discussion visit
> https://groups.google.com/d/msgid/weewx-user/2e0d1f8c-8531-401e-9bfd-f93123d8ec79n%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-user/2e0d1f8c-8531-401e-9bfd-f93123d8ec79n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/weewx-user/CAH3iRs3bZjgNuEFdXWg8VWG6xqHk7MeUtcvndKHLe_-44utUBw%40mail.gmail.com.
