Looking through the capture log, the LOOP packets are arriving out of sequence. This causes accumulators with good data to be thrown away.
Details: The way the engine works, it retains an accumulator for the immediate archive interval. As LOOP packets arrive, they are compared to the interval in the accumulator. If the timestamp of the packet is inside the interval, then it is added. If it is outside, an OutOfSpan exception is raised by the accumulator. The engine catches this and sets the accumulator aside and creates a new accumulator with an archive interval that includes the timestamp of the new LOOP packet. The packet is then added to the new accumulator. LOOP packets continue to arrive and get incorporated into the accumulator. Eventually, a packet arrives with a timestamp far enough into the future (the "archive delay", typically 15 seconds), that a record is extracted out of the old accumulator and put into the database. The old accumulator is discarded. Now consider an out of sequence packet with a timestamp in the previous archive interval. Its timestamp will be outside what the present accumulator is prepared to accept, so an OutOfSpan exception will be raised. Note that while the intention is to detect when it's time to move on to a new archive interval, a timestamp in the past can have the same effect. The OutOfSpan exception will cause yet another new accumulator to be created, this time with a timestamp in the past. You can see how it would be easy to discard perfectly good accumulators to make space for the out of sequence packets. What to do? I see 3 options: 1. The best solution is to make sure the timestamps of the LOOP packets arrive in monotonically increasing order. I thought we required this somewhere in the documentation, but if we did, I can't find it. 2. Right now, there are two choices when checking a LOOP packet timestamp: it's either in the accumulator's interval, or it is not. Instead, we would allow 3 states: if it's before the interval, discard it. If it's in the interval, incorporate it. If it's after the interval, create a new accumulator (as described above). 3. While the old accumulator is sitting around waiting for the archive delay passes, we could put it in play and allow the out of sequence packet to be added to it. This would give a 15 second grace for LOOP packets to catch up. I'm not keen on this one because it feels ripe for unintended consequences. Is solution #1 possible? -tk On Thu, Oct 9, 2025 at 9:03 AM matthew wall <[email protected]> wrote: > On Thursday, October 9, 2025 at 11:31:04 AM UTC-4 [email protected] wrote: > > The only thing I can think of: are you sure that the archive records for > 09:35 and 09:40 include LOOP packets for the missing observation types? For > example, does it include a packet containing p22_current? You're only > showing packets that would go into the previous record (09:30). > > > capture from two archive intervals is attached. > > these are the two archive records: > $ grep REC tigo-capture-251009 > REC: 2025-10-09 11:45:00 EDT (1760024700) 'dateTime': '1760024700', > 'identifier': '4610.25', 'interval': '5.0', 'p25_current': '0.55', > 'p25_dc_dc_duty_cycle': '1.0', 'p25_rssi': '192.0', 'p25_temperature': > '32.1', 'p25_voltage_in': '38.6', 'p25_voltage_out': '38.3', 'usUnits': '1' > REC: 2025-10-09 11:50:00 EDT (1760025000) 'dateTime': '1760025000', > 'identifier': '4610.25', 'interval': '5.0', 'p25_current': '0.535', > 'p25_dc_dc_duty_cycle': '1.0', 'p25_rssi': '192.0', 'p25_temperature': > '32.1', 'p25_voltage_in': '37.75', 'p25_voltage_out': '37.5', 'usUnits': '1' > > but you can see many other loop packets: > > LOOP: 2025-10-09 11:44:45 EDT (1760024685) 'dateTime': '1760024685.50201', > 'identifier': '4610.32', 'p32_current': '0.76', 'p32_dc_dc_duty_cycle': > '0.9372549019607843', 'p32_rssi': '147.0', 'p32_temperature': '37.0', > 'p32_voltage_in': '38.65', 'p32_voltage_out': '35.5', 'usUnits': '1' > LOOP: 2025-10-09 11:44:43 EDT (1760024683) 'dateTime': > '1760024683.500156', 'identifier': '4610.31', 'p31_current': '0.6', > 'p31_dc_dc_duty_cycle': '1.0', 'p31_rssi': '135.0', 'p31_temperature': > '35.5', 'p31_voltage_in': '38.65', 'p31_voltage_out': '38.5', 'usUnits': > '1' > LOOP: 2025-10-09 11:44:45 EDT (1760024685) 'dateTime': '1760024685.50201', > 'identifier': '4610.31', 'p31_current': '0.82', 'p31_dc_dc_duty_cycle': > '1.0', 'p31_rssi': '135.0', 'p31_temperature': '35.5', 'p31_voltage_in': > '38.7', 'p31_voltage_out': '38.5', 'usUnits': '1' > LOOP: 2025-10-09 11:44:43 EDT (1760024683) 'dateTime': > '1760024683.500156', 'identifier': '4610.38', 'p38_current': '0.435', > 'p38_dc_dc_duty_cycle': '1.0', 'p38_rssi': '105.0', 'p38_temperature': > '33.4', 'p38_voltage_in': '39.05', 'p38_voltage_out': '38.6', 'usUnits': > '1' > LOOP: 2025-10-09 11:44:45 EDT (1760024685) 'dateTime': '1760024685.50201', > 'identifier': '4610.38', 'p38_current': '0.59', 'p38_dc_dc_duty_cycle': > '1.0', 'p38_rssi': '105.0', 'p38_temperature': '33.4', 'p38_voltage_in': > '39.1', 'p38_voltage_out': '38.7', 'usUnits': '1' > LOOP: 2025-10-09 11:44:43 EDT (1760024683) 'dateTime': > '1760024683.500156', 'identifier': '4610.30', 'p30_current': '0.595', > 'p30_dc_dc_duty_cycle': '1.0', 'p30_rssi': '132.0', 'p30_temperature': > '35.7', 'p30_voltage_in': '38.75', 'p30_voltage_out': '38.5', 'usUnits': > '1' > LOOP: 2025-10-09 11:44:45 EDT (1760024685) 'dateTime': '1760024685.50201', > 'identifier': '4610.30', 'p30_current': '0.82', 'p30_dc_dc_duty_cycle': > '1.0', 'p30_rssi': '132.0', 'p30_temperature': '35.7', 'p30_voltage_in': > '38.8', 'p30_voltage_out': '38.5', 'usUnits': '1' > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/weewx-user/2636ac0e-994e-4782-861e-e1dcdfdef211n%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/2636ac0e-994e-4782-861e-e1dcdfdef211n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEBbyVqkAopZQ2sXE79F709Ok7dLYeDz5XfP7pc7ESfmvw%40mail.gmail.com.
