I am in the full agreement with the proposed approach (risked to copy below,
let's see if email handles your diagrams):
* Single Engine handles multiple executions asynchronously, in non-blocking way.
* Persistence access only from API and/or Engine, Engine writes, API reads.
* Action runes don't talk to DB - it's very simple protocol and queue is
perfectly fine.
* This is scalable and as fault tolerant as underlying DB and Queue. Engine
restart is no loss of info with right persistense; ActionRunner restart is a
lost of Action, which can fail for 100 other reasons thus expected, and with
the right retry policy potentially recoverable.
I'll inline minor points in the etherpad.
API Engine Queue Database Workers
| | | | | | | |
-------> +-+ | | | | | | |
|S| | | | | | | |
|t| --> +-+ | | | | | |
|a| | | < - - - - - - - -> | | |
|r| | | - - - -> t - - - - - - - - - > +-+ |
|t| <-- +-+ | | | | | | |
<------- +-+ | | | | | | | |
| | +-+ <- r <- - - - - - - - - - +-+ |
-------> +-+ | | | <- - - - - - > R | |
|I| | | | - -> t - - - - - - - - - > +-+ |
|n| | | | - -> t - - - - - - - - - - - > +-+
|f| | +-+ | | | | | | |
|o| <- - - - - - - - - - - - > | | | | |
<------- +-+ | +-+ <- r <- - - - - - - - - - -+-+ | |
| | | | <- - - - - - > R | | |
-------> +-+ | +-+ | | | | | |
|S| --> +-+ | | | | | | |
|t| | | <- - - - - - - - > | | | |
|o| <-- +-+ | | | | | | |
|p| | +-+ <- r < - - - - - - - - - - - - +-+
<------- +-+ | | | <- - - - - - > R | |
| | |%| | | | | |
| | +-+ | | | | |
| | | | | | | |
On Mar 31, 2014, at 4:09 AM, Kirill Izotov <[email protected]> wrote:
> I have an idea regarding engine design i want to share with you. But first,
> it seems like we need a small overview of the current implementations.
>
> I'm not sure how ML will react on a bunch of ASCII graphs, so here is an
> etherpad:
> https://etherpad.openstack.org/p/mistral-engine-overview-and-proposal
>
> What do you think, guys, is this the way to go?
>
> --
> Kirill Izotov
>
> _______________________________________________
> OpenStack-dev mailing list
> [email protected]
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
