Raymond created CAMEL-21413:
-------------------------------
Summary: Create a TransactionLog transaction log to provide
detailed tracking and performance metrics on end-to-end transactions
Key: CAMEL-21413
URL: https://issues.apache.org/jira/browse/CAMEL-21413
Project: Camel
Issue Type: Wish
Components: came-core
Reporter: Raymond
Attachments: TransactionStore_TrackAndTrace.jpg
*Wish*
The wish is for a transaction log store in Camel to provide detailed tracking
and performance metrics on end-to-end transactions across multiple
exchanges/routes, storing metadata, exchange data, and performance metrics in a
configurable, persistent registry to better monitor business processes.
*Background*
A lot of data in tracing and observability is on the exchange level. How many
exchanges are completed or failed, for example. In many cases, what I really
want to know if a transaction is successful or failed.
With a transaction I mean that the complete lifecycle of a message on Camel,
this
may contain many routes and exchanges. For example, when Seda or ActiveMQ is
used
a new exchange is created, but it's still the same transaction.
To have metrics and more insights on transactions, I would like to have a
transaction log (through a separate registry?). This is a separate store with
information about the transaction.
*Implementation ideas*
Below are few ideas to implement this wish:
_1. Initialize the store_
Setup a transaction log/store (either in memory or external store) through a
registry. This is to enable it and to configure the store.
_2. Transaction item_
A transaction item is a definition of a specific transaction. It may contain
the following items:
* {*}Name{*}: The name to indicate the transaction. Like "order" or "invoice".
This is so to say the overarching name of the integration/business process.
* {*}StartingPoint{*}: A routeId or groupId where the transaction starts
(triggers a new transaction).
* {*}CorrelationID(s){*}: One or more correlationId (headers) that can be set
just like a header/exchangePropery/variable, this contains for example the
orderID that is set on the first step of the transaction.
_3. Tracking transactions_
A transaction is probably best tracked by something like a breadcrumbId.
Preferably a transactionId. At least something that cannot be edited by users
(I sometimes see people change the breadcrumbId, to separate transactions into
multiple segments).
By default, only the start and end of a transaction is tracked. To get more
fine-grained control, one could also track intermediate points like a route, a
step, or a node.
The transaction is tracked during processing, and then finished on completion.
The whole counts as one "UnitOfWork".
_4. The transaction store_
The transaction store contains all the tracked items. Maybe there could be
divided into separate segments like:
# metadata
- name (name of the transaction)
- description (description of type of transaction)
- transactionId
- transactionHistory (similar to messageHistory)
- correlationId(s)
- timestamps
- exceptions
# exchange data
- headers, properties, variables
- bodyType
- body
etc
# performance metrics
- load (CPU), memory (RAM), threads
- number of transactions (completed, pending, failed)
- throughput
- response time
- total process time of an transaction
- average
- etc
It should be possible to configure which of these items the user what want to
store. By default, only metadata.
_5. Transaction API's_
There need to be an API to configure and operate the transaction store.
(including JMX), similar to backlogtracer.
*Production*
Some of the solutions I saw are only available during development or test, but
the need is especially big for production environment. A solution needs to keep
in mind that this would run for billion of messages. The flexibility to turn it
easily on and off.
For example, by default, the body of a message doesn't need to be stored. Or
only the first 32kb need to be stored. But for specific case users, one to
enable this on the fly for a specific transaction.
Transactions are probably needed persistent, so either need a messagestore
(disk) similar to brokers or an external database. I'm not sure what the best
stores are for this kind of data. A normal SQL database like Postgres, a search
database like Elastic, a metrics database like Prometheus or key/value store
like Redis?
*End users*
The idea is that end users have a good insight in what is going on in the
integration layer. So that they can question like the following:
* How many order transactions are processed?
* Give me a specific order by orderId that custom says he didn't receive?
* How many failed messages are there? Which ones did go wrong, where did it go
wrong, and what is the exact error?
* Can we replay the failed order?
* Where is the bottleneck on the system
*Conclusion*
This is just an idea/wish, but something I see coming back from the start of my
career as integration specialist (around 2007), until today.
Most consultancy firms or end customers have built something themselves, I see
with names like "Track & Trace", "MessageStore", "ESB Collector",
"Transactions", "Control center".
I also built a solution myself, and are happy to share this solution, or others
I saw. A more generic solution to monitor (track & trace) transactions of
business processes would be very welcome.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
