Thanks, Tyler. "Deduping (i.e. normal conflict resolution) happens per-page"
Are the queries sent from the coordinator to other nodes sequencing through partitions in token order and that's what allows the coordinator to dedupe with just a single page at a time? IOW, if a target node responds with a row from token t, then by definition there will be no further rows returned from that node with a token less than t? And if I understand all of this so far, this means that for 3.x COUNT (and other aggregate functions) are "safe but may be slow" (paraphrasing Sylvain.) Is this for 3.0 and later or some other 3.x (or even some 2.x)? There remains the question of recommended usage for COUNT. I think my two proposed guidelines remain valid (ignoring the old timeout issue), with the only remaining question about how large a row count is advisable for "decent" request latency. 1,000? 10,000? Granted, it depends on the specific data and hardware, but I'm thinking that the guidance should be that you should only use COUNT(*) for no more than "low thousands" of rows unless you are willing to accept it both being very slow and very disruptive to normal cluster health. IOW, it's more like a batch analytics operation than a real-time operation. An occasional administrative query to measure table size should be okay, but common use for OLTP should be restricted to relatively narrow slices or row counts... I think. Feedback welcome. The upcoming support for 2GB partitions will be interesting, but the same guidance should cover, I think. Maybe the numeric upper bound might be a bit higher since only a single partition is involved, but if processing many thousands of rows will remain time consuming, it sounds like that should be treated more as a batch-style OLAP operation rather than a real-time OLTP operation... I think. Thanks. -- Jack Krupansky On Tue, Apr 19, 2016 at 12:04 PM, Tyler Hobbs <ty...@datastax.com> wrote: > > On Tue, Apr 19, 2016 at 9:51 AM, Jack Krupansky <jack.krupan...@gmail.com> > wrote: > >> >> 1. Another clarification: All of the aggregate functions, AVG, SUM, MIN, >> MAX are in exactly the same boat as COUNT, right? >> > > Yes. > > >> >> 2. Is the paging for COUNT, et al, done within the coordinator node? >> > > Yes. > > >> >> 3. Does dedupe on the coordinator node consume memory proportional to the >> number of rows on all nodes? I mean, you can't dedupe using only partition >> keys of the coordinator node, right? What I'm wondering is if the usability >> of COUNT (et al) is memory limited as well as time. >> > > Deduping (i.e. normal conflict resolution) happens per-page, so in the > worst case the memory requirements for the coordinator are RF * page size. > > > > > -- > Tyler Hobbs > DataStax <http://datastax.com/> >
