On Fri, Nov 15, 2013 at 7:04 PM, Stephen <[email protected]> wrote:
> > On Tue, Nov 12, 2013 at 8:07 PM, Mathieu Simard < > [email protected]> wrote: > >> Since there is no appengine solution available such as the Redis atomic >> list, I'm left wondering how to implement a cost effective rolling median. >> Has anyone come up with a solution that would be more convenient than >> running a redis instance on Google Compute Engine? >> > > - batch data in front-end instance memory > - flush data to a pull queue every time window > - run a task every time window to gather all data batches from pull queue > - merge data, compute moving median, write result to data store > > Instances will be started as more data is submitted. Tune the frequency > of the calculation so that the size of the pending data batches does not > overwhelm a single task. > As a refinement of the above: - serve a zero-byte static file at /ok - have your clients request /ok?k=foo&v=42 - periodically parse the request log: https://developers.google.com/appengine/docs/go/logs/ The advantage here is that: - you don't have to pay for front-end instance hours - your clients are served quickly by Google's edge servers - few moving parts in the critical path that can fail You could push your rolling medians into Big Query for analysis, which might become economic as you're dealing with only a sample of the original data. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
