Claus Ibsen created CAMEL-26790:
-----------------------------------
Summary: camel-chaos - an experiment that cannot tell does not
pass: a fault that never fired, failures outside its expectations, an
expectation of failures in a route with a dead letter channel
Key: CAMEL-26790
URL: https://issues.apache.org/jira/browse/CAMEL-26790
Project: Camel
Issue Type: Improvement
Components: camel-jbang
Reporter: Claus Ibsen
Assignee: Claus Ibsen
Fix For: 4.24.0
Found in the recorded runs of {{camel cmd chaos-explore}} (CAMEL-26760) on the
chaos example shop, read afterwards. Each made an experiment PASS that said
nothing:
# *A fault that never fired*: {{route=payment error rate=0.05}} fired 0 times
over about 40 messages (chance, 12%), and the experiment passed. With no fault
fired, the expectations say nothing about the fault: the verdict should be
INCONCLUSIVE, with that reason.
# *Failures outside the expectations*: an experiment expecting
{{refunds:maxTime<=30s}} passed while 2 refunds failed. The experiment should
list the routes that failed and that no expectation of it counts ({{failed}}),
so neither the AI nor a person misses them.
# *An expectation of failures in a route with a dead letter channel*:
{{expect=parked:failed<=0}} on a route whose error handler is a dead letter
channel is always met, as the dead letter channel handles what fails there (7
orders that could not be parked were only logged). Adding such a rule should
warn, as for a rule that matches nothing, and say that {{handled}} counts what
the dead letter channel handled.
The same pattern as the other fixes of these runs: Camel catches an experiment
that cannot tell, and says why, so the model corrects itself.
_Claude Code on behalf of Claus Ibsen_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
