Gerrrr commented on code in PR #28: URL: https://github.com/apache/hunter/pull/28#discussion_r1929948727
########## docs/ANALYZE.md: ########## @@ -0,0 +1,46 @@ +# Finding Change Points + +``` +hunter analyze <test>... +hunter analyze <group>... +``` + +This command prints interesting results of all +runs of the test and a list of change-points. +A change-point is a moment when a metric value starts to differ significantly +from the values of the earlier runs and when the difference +is consistent enough that it is unlikely to happen by chance. +Hunter calculates the probability (P-value) that the change point was caused +by chance - the closer to zero, the more "sure" it is about the regression or +performance improvement. The smaller is the actual magnitude of the change, +the more data points are needed to confirm the change, therefore Hunter may +not notice the regression after the first run that regressed. + +The `analyze` command accepts multiple tests or test groups. +The results are simply concatenated. + +#### Example + +> [!TIP] +> See [hunter.yaml](../examples/csv/hunter.yaml) for the full example configuration. + +``` +$ hunter analyze local.sample --since=2024-01-01 +INFO: Computing change points for test sample.csv... Review Comment: Linked used data in [e47e1c0](https://github.com/apache/hunter/pull/28/commits/e47e1c0972527da835cb95fb332ef6acefb630a2). We can replace that with the Tigerbeetle data, I don't have a preference. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hunter.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
