To tell the truth, I don't like the use of cglib in this case either, it is per-request of making callback input available to IDE's intelli-sense editing and be friendly to automatic code-refactoring through IDE (Eclipse) which we do all the time.
I'm fine with us to go with other standard approaches, my concern is the timing. Kelven On 9/5/13 3:25 PM, "Darren Shepherd" <darren.s.sheph...@gmail.com> wrote: >Kelvin, > >I really have some concerns about the entire async framework that was put >into place as part of the ipc framework. I don't know what VM sync is so >I can't really comment too much on that. > >Basically my concern with the async framework is that it's yet another >custom ACS thing. The use of cglib in the callbacks (or futures?) is to >obscure and is not clear to a new person coming into ACS (for example, >me). > >If we wish to do async stuff it should be based off of java 5 futures or >guava's ListenableFutures. Those APIs and libraries are far more natural >to java devs in the wild. Or we can get inspiration from scala and akka >on how to do async well in java. In short, if we have a real use case >for async, then I'd prefer we use an off the shelf library. > >In general I will lean towards most code being synchronous. To answer >Chiradeeps comment on how do we do insane scale. That just requires true >horizontal scalability of the management stack (which I'm not too sure if >that's been fully proven beyond 3 servers and 30k hypervisors, but that's >something we can address) and a really big database and somebody who >enjoys really large failure domains. > >Darren > >On Sep 5, 2013, at 2:51 PM, Kelven Yang <kelven.y...@citrix.com> wrote: > >> New VM sync and job flow handling are moving towards event-driven model >> which is async natural. Not sure about the details on how storage >> framework is using async mechanism, but argument on callback to assume >> memory state does not stand as strong as it might be, since arguments in >> any regular call pass information in memory the same way. >> >> Excessive callbacks (async method) usage pattern does encourage people >>to >> program in a wrong direction of async programming, I would prefer to >> substitute callback with event signaling and maintain/access >>flow-context >> through separated facility with well defined API, this can prevent >> developers to pass too much information directly and create a >> tight-coupling pattern. >> >> However, I'm a little concerning about yet another storage refactoring >> right after its previous refactoring work is barely done. >> >> Kelven >> >> On 9/5/13 12:57 PM, "Chiradeep Vittal" <chiradeep.vit...@citrix.com> >>wrote: >> >>> +1 for removing complexity especially if the sync pattern is being >>>used on >>> top of the async pattern. >>> I see this behavior in the AgentManager.send as well -- even thought >>>the >>> AM >>> is capable of async, practically nobody uses it as such. >>> >>> But I guess the question will arise : what if I do want more than 10^n >>> long-running storage jobs >>> (cos my cloud is as successful as AWS :)) >>> >>> On 9/4/13 5:03 PM, "Darren Shepherd" <darren.s.sheph...@gmail.com> >>>wrote: >>> >>>> I've been reading over the storage code and have come to the >>>>conclusion >>>> that the async aspects of the storage framework should be removed. >>>> >>>> Whenever one introduces an async pattern you have to give a lot of >>>> consideration to its use, benefits, and impact. Within the context of >>>> ACS and given the current state of its code, I do not think it will be >>>> possible to realize any benefits of the current callback approach. >>>> Since nothing else in ACS uses callbacks, all of the async methods are >>>> essentially wrapped in synchronous calls. So nothing as it stands is >>>> actually async. >>>> >>>> Besides the current implementation, you need to conciser how you would >>>> expect an implementation of the storage framework to use the callback. >>>> The problem with callbacks is that they assume some in memory state. >>>> This means if the process/server crashes that state is lost. Many >>>>will >>>> say just serialize the callback to the DB, but that is very >>>>impractical. >>>> >>>> Since ACS doesn't actually stand in the data path, an async pattern >>>> won't really even allow it to have better performance. ACS is just >>>> waiting for some storage operation to happen. ACS can easily spawn >>>>1000 >>>> threads and have them all wait. If you were to get to this point, >>>>you'd >>>> find that downstream you'll most likely have issues as you have 1000 >>>> create template operations so its killing your filer. So you will >>>> throttle storage operations to a level that won't kill your >>>> infrastructure and that level is no where near the scalability limits >>>>of >>>> threads. >>>> >>>> The callbacks pattern really complicates the code and I see no real >>>> benefit. Instead of spending a lot of effort trying to make all of >>>>ACS >>>> async to make it beneficial, I'd say that effort should be spent on >>>> making ACS idempotent and crash-only. The point being, there's more >>>> beneficial things we can do with our time. >>>> >>>> Given that only solidfire implements the new framework (and ACS legacy >>>> too), I would assume its a simple things for Edison to just go and >>>> quickly change it non-async. >>>> >>>> Darren >>
