Claus Ibsen created CAMEL-26807:
-----------------------------------
Summary: camel-chaos - a score of a chaos exploration in its
report, to compare models and runs
Key: CAMEL-26807
URL: https://issues.apache.org/jira/browse/CAMEL-26807
Project: Camel
Issue Type: New Feature
Components: camel-jbang
Reporter: Claus Ibsen
Fix For: 4.25.0
The explorations of CAMEL-26760 were compared by a script that read their
journals: per run the experiments that could not tell (hollow: the faults never
fired, the route of the expectation had no messages, a loose expectation such
as failed<=5, failures of the route of the expectation that no expectation
counts), the gaps it found by a FAIL, and the share of the budget used. It
showed a weak spot that had been in three runs before anyone noticed
(CAMEL-26797).
The report of an exploration could carry this score, so a nightly run of
{{camel cmd chaos-explore}} on the same example and model (as the AI ladder of
the demos does) can be compared over time, and a change of the playbook, the
tools or the model measured.
_Claude Code on behalf of Claus Ibsen_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
