Hi Aleks,
Thank you for your initiative in proposing this system enhancement. I strongly support this change, as it addresses a critical need in our current architecture. The existing synchronous command handler facilitates the maker-checker workflow. It allowed direct storage of serialized commands (including request payload JSON) in the database, enabling easy retrieval and replay during checker approval. As we transition to the new approach, Are there proposed modifications to the maker-checker workflow itself? Additionally, the existing implementation includes a permission check (authorization) prior to invoking the handler, as well as a Hook Event Processor integrated into the control flow to invoke external APIs asynchronously. To ensure continuity of these functionalities, can we include these in the proposed command executors? This will help maintain critical auditability standards while adopting the updated architecture. Regards, Manoj On 2025/01/26 09:38:46 Aleksandar Vidakovic wrote: > On Sun, Jan 26, 2025 at 6:59 AM VICTOR MANUEL ROMERO RODRIGUEZ < > victor.rom...@fintecheando.mx> wrote: > > > Hello, > > > > I have been reading the discussion, the FISP and the GitHub PR... this > > must receive more feedback from the community. > > > > That's why we are here... thanks again for participating. > > > > > > It causes me some noise that the FSIP is focused on the API Rest layer and > > the PR is introducing a new header that could also be used for > > Idempotency... We can understand the reasons for having it in the Rest > > layer. But what about the idempotency that is being used in the batch/event > > processing? > > > > Sorry for the noise... > I'm not really seeing the "focus" on the REST API layer... the proposal is > kept explicitly free of any assumption which web stack you are using... > this stuff could even work with Webflux. > Introducing a new header? There is not even an implementation for > idempotency in this proposal... maybe you are referring to the example in > the unit test? That stuff (everything in the "sample" package) is purely > for demonstration. > Could you maybe elaborate what problem you see concerning the batch > processing? > > > > > > I have found the implementation for sync and async commands... What about > > streaming? Also in the same package there are pipeline, executor, router .. > > What about notifications? My questions are for getting feedback from you if > > they are expected/discarded on this new proposed infrastructure. > > > > ... I would add to your list "what about non-blocking"? Well, the reason > why it's not in my proposal is, because right now we have a classic > synchronous blocking execution path in place upstream... and the first > step would be to get the whole thing gradually to a more performant > solution with clearly structured internal API. Another important thought > here is to avoid/reduce any additional learning curve; the community is > used to the classic "I call a function and get a result" paradigm of > programming; introducing more complex paradigms would need more time for > adoption. That being said: if the community decided tomorrow to go all in > for non-blocking, reactive, GRPC, [place your preference here] and > bulldozer things then I would happily participate... but my feeling is that > this is very unlikely. And before anything like this happens we would need > all these hard-wired references to JSON data structures from the business > logic services anyway. > > As for notifications: noted, good point. > And concerning feedback: I have no claim here that what I wrote in the PR > is set in stone... that's why we discuss it here (I hope more people join). > Things that I didn't have on the radar and that you and others find > indispensable can be of course added. > > > > > > I think that Apache Camel is a good tool... I don't see it as a vendor > > lock-in... well If do that (to see it as a vendor lock-in)... then what > > about spring boot itself? i.e. nowadays we have Quarkus and Micronaut... > > and I think it is more complex to move some Apache Fineract code to these > > frameworks or add plugins developed in these frameworks to Apache Fineract > > runtime. > > > > Again, the idea here is to keep this proposal as self contained as possible > to avoid having to decide on too many fundamental changes... hence, you see > no Apache Camel. That being said: I can write you an adapter in 10min that > runs the whole thing over Camel... but doesn't leak any details about Camel > being used to the rest of the code base. Don't see what the flaw would be > here to properly abstract implementation details? > > Is there a ready to use solution in Spring Boot for command processing > and/or CQRS? I think not... but let me know if I am missing something here. > What is out there is Axon which is a complete framework implemented ON TOP > OF Spring/Boot... but that thing would require us to jump on their internal > APIs and how they think things should be processed... personally I think > these guys thought about this subject a bit longer and have a way more > complete solution... but again, given the need for backwards compatibility > and the requirement to be not (too) disruptive to any other upstream > development: very unlikely that such a fundamental change would happen and > cause a major refactoring fest. > > I'm not so sure why Quarkus and Micronaut are mentioned here... yes, very > nice and capable modern frameworks... but they also have no specific notion > of command processing, but are generic frameworks like Spring/Boot... my > intention is not to change any fundamental underlying frameworks (give and > take they have similar features)... again, I'd like to keep this as small > and self contained as possible. > > > > > > Happy to read your feedback. > > > > Regards > > > > Victor > > > > > > > > > > El jue, 23 ene 2025 a las 23:40, James Dailey (<jdai...@apache.org>) > > escribió: > > > >> On Thu, Jan 23, 2025 at 9:20 AM Aleksandar Vidakovic > >> <chee...@monkeysintown.com> wrote: > >> > > >> > ... thanks James for the input... I'll try to answer your last couple > >> of questions from my perspective (read: opinionated... take with a pinch of > >> salt): > >> > >> JD: Aleks - thank you. I always learn something from this back and > >> forth with you. > >> > > >> > too clever: the current implementation I suggest that anyone tries to > >> draw a sequence diagram that explains the flow of execution and make it fit > >> on one page vs the new proposal will most likely contain less than a > >> handful of lines. You can apply the same if you take lines of code as a > >> metric... overall the new proposal has less than 50 lines of code that are > >> relevant (I don't know the number for upstream, but I think it's safe to > >> say it's more). If we assume that we can achieve the same results with less > >> code then I think the answer is easy here > >> > maintainability: well, see above... the current solution is not > >> documented at all and I am pretty sure I am not alone when I saw "I really > >> can't explain all the steps" (doesn't mean they are not necessary); what I > >> want to say is that the existing solution would really need a lot more > >> explanation than just "CQRS", I think that would be a fair requirement. > >> Admittedly, the new proposal also has no documentation (other than the wiki > >> page and what I wrote in this message). But: I think if I did write it it > >> can fit on one page (with diagrams), this module (it's a real one) has > >> (almost) no external dependencies (other than the frameworks that we use > >> anyway), it makes no assumption about any of the business logic that might > >> or might be passing through (existing implementation fails already there... > >> see CommandWrapper and the various entity IDs that are buried there... this > >> wrapper class should not be aware of anything it transports)... which > >> brings me back to the point of less code which is I think from a > >> maintenance point of view preferrable > >> > >> JD: When I say "overly clever" that is in contrast to simplicity > >> through elegant design. A favorite quote "There are two ways of > >> constructing a software design: One way is to make it so simple that > >> there are obviously no deficiencies, and the other way is to make it > >> so complicated that there are no obvious deficiencies. The first > >> method is far more difficult." I think if you are aiming for > >> something simple enough to have obviously no (or much fewer) > >> deficiencies, that is, an improvement. But, could you write some > >> documentation about the concept? It should be simple to describe "on > >> paper", yes? > >> > >> > > >> > Apache Camel: ... disclaimer, I really like that framework and used it > >> on a ton of occasions. That being said: choosing a framework is a > >> commitment pretty much like a vendor lock-in. Depending on how you > >> integrate a framework like Camel (this will be more than a JAR file and you > >> can either hide the fact you use Camel from the rest of your app or you > >> fully expose it...) upstream means if for some reason it turns out that > >> Camel is not a good choice or the community doesn't want yet another > >> dependency then we might find ourselves in a refactoring fest to revert > >> things. If you look closely in the proposed sources you will see that first > >> of all there are Java interfaces that propose a contract on how to wire > >> things together... and there not many... which leaves a lot of room for > >> actual implementations (Camel or something else). In fact, 3 or 4 years ago > >> I actually created a drop-in replacement for the upstream > >> SynchronousCommandProcessing service and ran Camel behind the scenes and > >> was actually very happey with the outcome. When I did this there were > >> basically 2 relevant functions that needed to be taken care of. Today there > >> is a lot more going on there and I am not so sure if you could just drop-in > >> Camel effortlessly with the current incarnation of the command processing > >> service. > >> > >> JD: Ok. I can buy not wanting another dependency, but only if our > >> level of effort is relatively small ongoing. Otherwise we are taking > >> on code maintenance for our "own thing" when a perfectly suited > >> solution is in the same software foundation. > >> > >> > Asking Apache Camel's community for opinion: well, can't hurt... they > >> do stuff like this literally every day, so I am pretty sure whatever we'll > >> exchange with them will be very informative. But that doesn't relieve us > >> from deciding if you want to go all in on Apache Camel it would be anyway a > >> good practice to abstract these implementation details away (aka hide to > >> the rest of Fineract that you are using Camel). If that is the case then we > >> need a contract (aka Java interface). The one that is there won't do it > >> anymore... without major rework... and that is the point. The proposal > >> intends to ensure a gradual non disruptive migration (not open heart > >> surgery) > >> > >> JD: Sure, that makes sense, you need to new Java interface... but > >> wouldn't it be better to spend a bit of time in design and validation > >> at this early stage. I think we're talking about a pretty significant > >> optimization from its location in the stack. Who should reach out? > >> > >> > whitepapers, alternatives: I think the first thing that Google or > >> ChatGPT searches will tell you is "use an existing CQRS framework"... and > >> this will most likely show you AxonIQ (a CQRS framework implemented with > >> Spring/Boot)... but that is then even more of a vendor lock in than using a > >> more generic solution like Camel... Axon will force us to use their > >> contracts (internal APIs, Java interfaces etc.), in short: refactor fest, > >> disruptive. There are other low level "solutions" (like LMAX Disruptor) > >> that are somewhat in the vicinity of this type of application, but require > >> work, to my knowledge there is nothing out there we could just magically > >> drop and use without any refactoring. Disclaimer: in one of the 3 drop-in > >> implementations of the proposed command processing I am actually using LMAX > >> Disruptor... its implementation details just don't leak into the rest of > >> the system > >> > Spring Boot 3 compliance: yes (buzzword drop: "auto-configuration") > >> > >> JD: Excellent > >> > >> > cutting edge: not sure how to read this here... is this meant as a > >> requirement or as an argument against the adoption of the proposal as in > >> "too experimental"... as I've written the code I am obviously biased so I > >> leave that to the community to decide and come up with improvements and/or > >> alternatives/arguments if someone doesn't agree > >> > >> JD: Yep. The ambiguity is on purpose - cutting edge can be great in > >> getting results, or it can make you bleed. > >> > >> > > >> > Let me know if I skipped something, made an error or was not clear > >> enough. > >> > >> JD: Very clear. Now, before this code is committed, I would also > >> like to be sure we have a sensible way of documenting the progress so > >> that if we are doing a release, we make note of how much of the code > >> base is using the new methods. I also think we should discuss this in > >> context of the next release. (coming up soon). > >> > >> > > >> > On Thu, Jan 23, 2025 at 5:00 PM <jdai...@apache.org> wrote: > >> >> > >> >> Thanks for bringing this to the list. It looks to be a very low level > >> (in the stack) and therefore, highly impactful. I was there when the > >> decision was made to adopt this pattern and > >> SynchronousCommandProcessingService as a flexible improvement to the > >> existing CQRS. I remember asking some questions, but this was and is, > >> beyond my direct experience. > >> >> > >> >> > >> https://cwiki.apache.org/confluence/display/FINERACT/FSIP-5:+New+command+processing+infrastructure > >> >> > >> >> What I do know is that we should be deliberate with this process, and > >> I appreciate your write up on wiki. Definitely other architects here > >> should take a look. > >> >> > >> >> At times over the past decisions - it feels to me that we try to be > >> "too clever", and this creates a problem with maintainability. I'd like to > >> make sure we understand the alternatives as we dig into this. You raised > >> Apache Camel as an option - would it be worth it to ask someone over in > >> that project to comment on this? Is there some whitepaper or comparison > >> out there between the alternatives available? Is this consistent with > >> Spring Boot 3 ? Is this on the cutting edge? > >> > > >
