What is CS? On Thu, Apr 7, 2016 at 10:03 AM Kevin Burton <bur...@spinn3r.com> wrote:
> I have a paging model whereby we stream data from CS by fetching 'pages' > thereby reading (sequentially) entire datasets. > > We're using the bucket approach where we write data for 5 minutes, then we > can just fetch the bucket for that range. > > Our app now has TONS of data and we have a piece of middleware that > filters it based on the client requests. > > So if they only want english they just get english and filter away about > 60% of our data. > > but it doesn't support condition pushdown. So ALL this data has to be > sent from our CS boxes to our middleware and filtered there (wasting a lot > of network IO). > > Is there away (including refactoring the code) that I could push this this > into CS? Maybe some way I could discovery the CS topology and put daemons > on each of our CS boxes and fetch from CS directly (doing the filtering > there). > > Thoughts? > > -- > > We’re hiring if you know of any awesome Java Devops or Linux Operations > Engineers! > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > blog: http://burtonator.wordpress.com > … or check out my Google+ profile > <https://plus.google.com/102718274791889610666/posts> > >
