additionally, a certain number of the threads in each stage could be restricted from serving the low-priority queues at all, say 8/32 or 16/32 threads, to further ensure processing availability to the higher-priority tasks.
On Wed, Jan 16, 2019 at 3:04 PM Carl Mueller <carl.muel...@smartthings.com> wrote: > At a theoretical level assuming it could be implemented with a magic wand, > would there be value to having a dual set of queues/threadpools at each of > the SEDA stages inside cassandra for a two-tier of priority? Such that you > could mark queries that return pages and pages of data as lower-priority > while smaller single-partition queries could be marked/defaulted as normal > priority, such that the lower-priority queues are only served if the normal > priority queues are empty? > > I suppose rough equivalency to this would be dual-datacenter with an > analysis cluster to serve the "slow" queries and a frontline one for the > higher priority stuff. > > However, it has come up several times that I'd like to run a one-off > maintenance job/query against production that could not be easily changed > (can't just throw up a DC), and while I can do app-level throttling with > some pain and sweat, it would seem something like this could do > lower-priority work in a somewhat-loaded cluster without impacting the main > workload. > > >
