henrikingo commented on PR #150:
URL: https://github.com/apache/otava/pull/150#issuecomment-4352743720
Ok thanks!
Yes so the test case you supply, and also the ones in the patch itself, do
match familiar behavior: e-divisive gets more volatile and therefore sensitive
towards the ends of a segment. A good way to illustrate this is to use your
repro from above, and modify it to have the outlier spike in the middle:
series[151] = 150.0
At least in my attempts, this will never be marked as a change point.
And this is also why the original e-divisive requires change points to be a
minimum amount of data points away from each other. I'll use `min_segment_len`
as you've done in the patch.
The trade off is that:
1. A newly introduced regression can only be detected after
`min_segment_len`/2 additional data points have accumulated. For example if
min_segment_len=10, and you run benchmarks in the nightly build, then you need
to wait 5 days and nights before you can expect to find a change point.
2. Two nearby changepoints, only one of them can be found. There needs to be
min_segment_len/2 points between two change points.
As for 1 I guess this is generally bad for everyone, the only argument for
it is that the results at the very end of a segment are not stable and I think
the paper even says that the proof only works in the middle of the segment, and
ends must be surpressed.
2 is subjective. For Datastax, the whole point of their modifications is
that they want the algorithm to find and flag 2 changes that are close to each
other. You otoh say it is fine, as long as one point is marked, a human is
alerted and see the other change point too. Both of these are valid opinions.
In conclusion: Adding a new option is precisely the right solution.
Before we proceed with this PR, I would like to introduce a new composition
of the algorithm. Currently we have
https://github.com/apache/otava/blob/1ceb1537a49d516cf6cef9a79bea6926002082f8/otava/analysis.py#L294
and
https://github.com/apache/otava/blob/1ceb1537a49d516cf6cef9a79bea6926002082f8/otava/analysis.py#L280
...the latter attempts to be an implementation of the literal Matteson &
James algorithm as described in their publication. Generally your modification
makes more sense against this latter variant of the algorithm. Adding it to the
current default variant you are kind of fighting against behavior thathe algo
was specifically designed to do.
So, I would like to add a variant of the algorithm, let's call it
`--deterministic-edivisive` variant, which is like `--orig_edivisive` but uses
the Student T-test instead of the random permutations tester.
Then your patch can be added both to --orig_edivisive and
--deterministic-edivisive and when doing so we are actually just adding back a
parameter that is in the original paper anyway.
And to remain backward compatible, we default this to 0, but you would set
it to e.g. 3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
