Claus Ibsen created CAMEL-26797:
-----------------------------------
Summary: camel-chaos - an experiment does not pass while the route
of its expectation failed, and a lost expectation between two routes that are
not a hand-off warns
Key: CAMEL-26797
URL: https://issues.apache.org/jira/browse/CAMEL-26797
Project: Camel
Issue Type: Improvement
Components: camel-jbang
Reporter: Claus Ibsen
Assignee: Claus Ibsen
Fix For: 4.24.0
Found in a run of {{camel cmd chaos-explore}} (CAMEL-26760) on the chaos
example (apache/camel-jbang-examples#134, which adds a refunds route that calls
the payment service over HTTP).
The model wrote {{expect=refunds->payment:lost<=0}} for "refunds handles a 503
from payment", and the experiment passed while 7 refunds failed:
# {{a->b:lost}} counts the messages route a completed that route b did not
complete (CAMEL-26726). That only means something for a hand-off (orders hands
to shipping over seda). Refunds does not hand off to payment: it calls it and
waits, so a 503 fails the refund in refunds, and a failed exchange is not
counted by lost at all. Payment also answered the orders (13 completed in
refunds, 57 in payment), so the expectation was 0 lost by construction.
# The experiment then had faults that fired, a met expectation, and a route
with messages, so it passed. The 7 failures were only listed under "also
failed, in no expectation".
The playbook already says to use {{->lost}} only for the hand-offs the
suggestions name; a small model does not always follow it (an earlier run had
the same mistake). So, as with the other checks of CAMEL-26790:
* Adding a rule with {{a->b:lost}} where route a does not hand messages to
route b (seda, vm, disruptor) warns, and says that {{expect=a:failed<=0}}
counts what fails in a.
* An experiment whose expectation is about a route that failed exchanges that
no expectation of it counts is INCONCLUSIVE, not PASS, with the reason and the
expectation that counts them.
_Claude Code on behalf of Claus Ibsen_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
