Yakov, thanks. Here are some thoughts from the Son of the Earth :)
> Another reason for not switching to striped pool completely is that stripes > are good for cache updates and single or batched get operations since they > reduce inter-thread communication and synchronization to minimum How did the striped pool show itself for transactional updates that involve 2 phase commit? Any benefits or degradation? > however I > think that we should run, for example, SQL queries in old fashioned pools - > first, they can fallback to scanning random data, second, we pause cache > updates in some partition for query execution time. Here we should decide what’s more important for us - throughout or latency. If to execute SQL queries in the striped pool using multiple partitioned threads then, for sure, it will affect latency of other operations that are sitting in the queue waiting for their time to be processed but, on the other hand, the overall throughput of the platform should be improved because the operations will be less halted all the time by synchronization needs. VoltDB decided to go for with the throughput while our current architecture is mostly latency based. — Denis > On Dec 15, 2016, at 11:22 AM, Yakov Zhdanov <yzhda...@gridgain.com> wrote: > > Denis, there was no discussion on dev list. Do not worry, Son of the Earth, > you are all right :) I thought I would send letter a bit later when I > finish the full scope of optimizations I have in plan. > > Currently, I use striped pool only for cache messages that are targeted to > some defined partition - cache update benchmarks showed very good growth in > throughput. > > I tried to switch public and system pool to striped mode, but I get a lot > of hangs on TC - sometimes due to issues in tests sometimes due to issues > in Ignite. For example, under some circumstances we can synchronously wait > in system thread for some message to come. When message handler gets to the > same stripe Ignite hangs. I have a list of such places discovered so far. > After I fix them I will try re-approach the pools. > > Another reason for not switching to striped pool completely is that stripes > are good for cache updates and single or batched get operations since they > reduce inter-thread communication and synchronization to minimum, however I > think that we should run, for example, SQL queries in old fashioned pools - > first, they can fallback to scanning random data, second, we pause cache > updates in some partition for query execution time. I was thinking of > splitting CPU resources between striped pool and system pool in some ratio. > I will follow up on this in a couple of days and will explain what has been > done and what I want to do. > > Vladimir, I hope I responded to your points. > > Thanks! > > --Yakov
