HI David,
Yesterday David Purdy wrote:
> Greetings,
>
> Is there a way to use HWPredict to filter errors (ie. outlier
> data) from the data stream, on the fly, at the moment it is being
> collected?
you could use rrdtool updatev to post your updates, then rrdtool
will tell you immediately what it wrote to the RRAs (if it did so).
cheers
tobi
>
> I have a 1-wire temperature sensor that is polled once a minute.
> Occasionally a piece of data is obviously a garbage/noise/outlier point, and
> ideally it should be discarded. I note that this is a GAUGE situation, but
> also, (normal) meteorological temperature data will be differentiable with
> respect to time: in the (rough) Calculus sense, there can't be any nasty
> cusps or vertices in the data.
>
> An example showing such a cusp is this ( the 19:24 data is borked) :
>
> <!-- 2014-01-26 19:19:00 CST / 1390785540 --> <row><v>1.2987500000e+01</v>
> <!-- 2014-01-26 19:20:00 CST / 1390785600 --> <row><v>1.2987500000e+01</v>
> <!-- 2014-01-26 19:21:00 CST / 1390785660 --> <row><v>1.2987500000e+01</v>
> <!-- 2014-01-26 19:22:00 CST / 1390785720 --> <row><v>1.2880211103e+01</v>
> <!-- 2014-01-26 19:23:00 CST / 1390785780 --> <row><v>1.2542394264e+01</v>
> <!-- 2014-01-26 19:24:00 CST / 1390785840 --> <row><v>6.7108108375e-01</v>
> <!-- 2014-01-26 19:25:00 CST / 1390785900 --> <row><v>1.0005696817e+01</v>
> <!-- 2014-01-26 19:26:00 CST / 1390785960 --> <row><v>1.2200000000e+01</v>
> <!-- 2014-01-26 19:27:00 CST / 1390786020 --> <row><v>1.1985339718e+01</v>
> <!-- 2014-01-26 19:28:00 CST / 1390786080 --> <row><v>1.1975000000e+01</v>
>
> Graphically, this might look like the spike at about 16:25 hrs :
> https://www.dropbox.com/s/zrvi15ez0zbqj2j/1wiretemps_showing_outlierspike.png
>
>
> My current and simplistic solution to this is to discard any data for which
> the rate of change exceeds a real-world limit.: 3.5 deg (F) per minute. For
> instance the fastest recorded temperature drop is 27.2 °C (49 °F) in 15
> minutes; Rapid City, South Dakota, 1911-01-10. So, currently I'm just testing
> the (absolute value of) temperature change over the last minute to see if it
> is less than 3.5 degrees. If so, then I'll assume it is good data, otherwise,
> I'll discard it.
>
> In effect, this requires the absolute value of the 1st derivative to be less
> than 3.5 (deg F/ min).
>
> Perhaps there is a more intelligent, sophisticated and built-in method for
> using the Holt-Winters methods (HWPredict), or perhaps the 2nd derivative as
> well?
>
> If so, could you provide some details on this, perhaps showing syntax and
> rpn-format?
>
> Thank you,
>
> Dave Purdy
>
>
>
>
> _______________________________________________
> rrd-users mailing list
> rrd-users@lists.oetiker.ch
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>
>
--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch t...@oetiker.ch ++41 62 775 9902 / sb: -9900
_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
