>
> The goal of the project is to develop an algorithm that automatically
> scales the cluster up and down based on the volume of data processed by the
> application.
By "scale the cluster up and down" do you mean:
1) adding/removing spark executors based on the load? How is that from the
dynami
e fetch the latest master. You are unable
> to create such a hive serde table without Hive support.
>
> Thanks,
>
> Xiao Li
>
>
> 2017-01-23 0:01 GMT-08:00 Shuai Lin :
>
>> Cool, thanks for the info.
>>
>> I think this is something we are going to change t
difference is whether the metadata is
>>> persistently stored or not.
>>>
>>> Thanks,
>>>
>>> Xiao Li
>>>
>>> 2017-01-22 11:14 GMT-08:00 Reynold Xin :
>>>
>>> I think this is something we are going to ch
no use. But I wonder if there are
other good reasons for the current logic. If not, I would propose to raise
an error when creating the table in the first place.
Thanks!
Regards,
Shuai Lin (@lins05)
Disclaimer: I'm not a spark guru, and what's written below are some notes I
took when reading spark source code, so I could be wrong, in which case I'd
appreciate a lot if someone could correct me.
> > Let me rephrase this. How does the SparkSQL engine call the codegen APIs
> to
> do the job of p
Sorry but I don't get the scope of the problem from your description. Seems
it's an improvement for spark standalone scheduler (i.e. not for yarn or
mesos)?
On Sat, Dec 3, 2016 at 4:27 AM, Hegner, Travis
wrote:
> Hello,
>
>
> I've just created a JIRA to open up discussion of a new feature that I
