New VM sync and job flow handling are moving towards event-driven model which is async natural. Not sure about the details on how storage framework is using async mechanism, but argument on callback to assume memory state does not stand as strong as it might be, since arguments in any regular call pass information in memory the same way.
Excessive callbacks (async method) usage pattern does encourage people to program in a wrong direction of async programming, I would prefer to substitute callback with event signaling and maintain/access flow-context through separated facility with well defined API, this can prevent developers to pass too much information directly and create a tight-coupling pattern. However, I'm a little concerning about yet another storage refactoring right after its previous refactoring work is barely done. Kelven On 9/5/13 12:57 PM, "Chiradeep Vittal" <chiradeep.vit...@citrix.com> wrote: >+1 for removing complexity especially if the sync pattern is being used on >top of the async pattern. >I see this behavior in the AgentManager.send as well -- even thought the >AM >is capable of async, practically nobody uses it as such. > >But I guess the question will arise : what if I do want more than 10^n >long-running storage jobs >(cos my cloud is as successful as AWS :)) > >On 9/4/13 5:03 PM, "Darren Shepherd" <darren.s.sheph...@gmail.com> wrote: > >>I've been reading over the storage code and have come to the conclusion >>that the async aspects of the storage framework should be removed. >> >>Whenever one introduces an async pattern you have to give a lot of >>consideration to its use, benefits, and impact. Within the context of >>ACS and given the current state of its code, I do not think it will be >>possible to realize any benefits of the current callback approach. >>Since nothing else in ACS uses callbacks, all of the async methods are >>essentially wrapped in synchronous calls. So nothing as it stands is >>actually async. >> >>Besides the current implementation, you need to conciser how you would >>expect an implementation of the storage framework to use the callback. >>The problem with callbacks is that they assume some in memory state. >>This means if the process/server crashes that state is lost. Many will >>say just serialize the callback to the DB, but that is very impractical. >> >>Since ACS doesn't actually stand in the data path, an async pattern >>won't really even allow it to have better performance. ACS is just >>waiting for some storage operation to happen. ACS can easily spawn 1000 >>threads and have them all wait. If you were to get to this point, you'd >>find that downstream you'll most likely have issues as you have 1000 >>create template operations so its killing your filer. So you will >>throttle storage operations to a level that won't kill your >>infrastructure and that level is no where near the scalability limits of >>threads. >> >>The callbacks pattern really complicates the code and I see no real >>benefit. Instead of spending a lot of effort trying to make all of ACS >>async to make it beneficial, I'd say that effort should be spent on >>making ACS idempotent and crash-only. The point being, there's more >>beneficial things we can do with our time. >> >>Given that only solidfire implements the new framework (and ACS legacy >>too), I would assume its a simple things for Edison to just go and >>quickly change it non-async. >> >>Darren >
