>
> So, I guess the lightweight API (no
> executorConfig) proposed originally might be the easiest solution that
> should mostly satisfy both current use cases. It would also be the easiest
> to throw away when/if we eventually get rid of the ExecutorConfig data
> stored within TaskConfig object.
Thanks for chiming in. I played with this idea a bit more and I tend to
agree that thrift lookup filter may be an overkill here. TBase does not
provide a way to iterate over all available fields, which makes this design
even harder to implement.
Also, the complexity/gain tradeoff is not ideal here
My personal preference would be to identify the different use cases, and
create API calls for those (i suspect there are relatively few). I'm not
fond of altering the thrift response schema on the fly based on request
parameters (e.g. SELECT x FROM). This makes for client and server code
that is
Small correction. TBase does not support lookup by field names, so the
format would have to be a list of field IDs:
// Defines a set of thrift paths to be populated when returning
ScheduledTasks.
// Single path format is a list of valid field IDs representing thrift
query path.
// Valid examples:
Bumping up this thread as pagination does not solve client cases where all
tasks must be pulled for evaluation. Example: SLA commands in aurora_admin.
I would like to explore the #2 idea proposed by David and subsequently
outlined by Bill/Suman.
Specifically, I am proposing to add an optional for
On Fri, May 30, 2014 at 11:22 AM, Suman Karumuri wrote:
> Looks like my earlier mail has bounced. Sending it again.
>
> Can you please add some explanation to AURORA-471 explaining where the time
> is being spent with some data and some micro benchmarks. As is it is
> unclear why thrift.flush sho
The new API can be a flagged version of the getTasksStatus API also.
On Fri, May 30, 2014 at 11:27 AM, Bill Farner wrote:
> >
> > I feel that we should drop the extra information from the
> > getTasksStatus first (if the client is unaffected by the change)
>
>
> The client would indeed be affec
>
> I feel that we should drop the extra information from the
> getTasksStatus first (if the client is unaffected by the change)
The client would indeed be affected by that (config diffing when performing
job updates). However, there's nothing stopping us from defining a new RPC
and message to d
Looks like my earlier mail has bounced. Sending it again.
Can you please add some explanation to AURORA-471 explaining where the time
is being spent with some data and some micro benchmarks. As is it is
unclear why thrift.flush should be taking so long. Is the default
implementation of TServlet.do
Great. +1 from me.
On Tue, May 27, 2014 at 7:39 PM, David McLaughlin wrote:
> Pagination would be a no-op to the client because it would be opt-in only,
> so it would continue to fetch all the tasks in one request.
>
> But you raise a good point in that presumably the client is also going to
> b
Pagination would be a no-op to the client because it would be opt-in only,
so it would continue to fetch all the tasks in one request.
But you raise a good point in that presumably the client is also going to
be blocked for several seconds while executing getTasksStatus for large
jobs. Making the
Interestingly, when we first expanded getTasksStatus, I didn't like the
idea, because I thought it would have exactly this problem! It's a *lot* of
information to get in a single burst.
Have you checked what effect it'll have on the command-line client? In
general, the command-line has the context
As outlined in AURORA-458, using the new jobs page with a large (but
reasonable) number of active and complete tasks can take a long time[1] to
render. Performance profiling done as part of AURORA-471 shows that the
main factor in response time is rendering and returning the size of the
uncompresse
13 matches
Mail list logo
