Re: [E] Re: Using Quantile sketches for additive metrics

2022-05-24 Thread vijay rajan
Hi Alex, You are right. I assumed that quantiles would have an intersection like Theta Sketches but they don't. If it did based on transaction ID, it would have been cool. So many traditional data mining & predictive analytics algorithms could be reimplemented with sketches. Thanks for your insi

Re: [E] Re: Using Quantile sketches for additive metrics

2022-05-23 Thread Alexander Saydakov
This seems to be a convoluted way of computing the sum of all values. This is an additive metric, easy to compute exactly, no sketches needed. On Sun, May 22, 2022 at 5:56 AM vijay rajan wrote: > Hi folks (And Lee), > > I think I have found what I was looking for in quantile sketches though I >

Re: [E] Re: Using Quantile sketches for additive metrics

2022-05-22 Thread vijay rajan
Hi folks (And Lee), I think I have found what I was looking for in quantile sketches though I am not able to formulate error bounds for the same. I should have raised a PR request but I am going to write the code here. The code below estimates the volume of the quantile sketche based on the exampl

Re: [E] Re: Using Quantile sketches for additive metrics

2022-05-03 Thread vijay rajan
Thanks Will. Please find my reply in-line below. But just to stay in line with my original question of a sketch for additive metrics, is that I can use such a sketch for on-the-fly aggregation by storing one such sketch per "dimension=value" pair without having to go to the table for aggregation.

Re: [E] Re: Using Quantile sketches for additive metrics

2022-05-02 Thread vijay rajan
Hi Will, Thanks for your response. I will send my clarifications in a day or two. Please do look at my detailed explanation & look at the datasets and results that I have shared. You should understand what I am trying to do. Essentially, an event_Id is a uuid for an event. A click stream will hav

Re: [E] Re: Using Quantile sketches for additive metrics

2022-05-02 Thread Will Lauer
OK, this is interesting. I've got some concerns and questions that I've put inline below. Will Will Lauer Senior Principal Architect, Audience & Advertising Reporting Data Platforms & Systems Engineering M 508 561 6427 Champaign Office 1908 S. First St Champaign, IL 61822 On Mon, May