Thanks Reynold.
On 03-Jan-2018 10:43 pm, "Reynold Xin" wrote:
> It hasn't been compiled yet, but you can look up all the features on JIRA
> by setting a filter on fixed versions. Usually the release manager compiles
> the list when it is towards the end of the release cycle (coming up soon).
>
>
There's been discussion going on in various PRs about what committers do, are
expected to do, and how they get coordinated; a general conclusion to these is
"this should be covered in the developer list"
Here then, are the 3 PRs where this has surfaced.
[SPARK-22026][SQL] data source v2 writ
The default implementation is a recursive treewalk, though HDFS and ADL both
push the work out to the remote system for performance.
If odd numbers are coming back on getContentSummary() against HDFS, then it's a
bug there. Though if its Jenkins test runs against the local FS, then it's in
the
Hey all, especially those working on the k8s stuff.
Currently we have 3 docker images that need to be built and provided
by the user when starting a Spark app: driver, executor, and init
container.
When the initial review went by, I asked why do we need 3, and I was
told that's because they have
This is neat. With some code cleanup and as long as users can still use
custom driver/executor/init-container images if they want to, I think this
is great to have. I don't think there's a particular reason why having a
single image wouldn't work. Thanks for doing this!
On Fri, Jan 5, 2018 at 5:06
We should definitely clean this up and make it the default, nicely done
Marcelo !
Thanks,
Mridul
On Fri, Jan 5, 2018 at 5:06 PM Marcelo Vanzin wrote:
> Hey all, especially those working on the k8s stuff.
>
> Currently we have 3 docker images that need to be built and provided
> by the user when
Awesome, less is better
Mridul Muralidharan 于2018年1月6日周六 上午11:54写道：
>
> We should definitely clean this up and make it the default, nicely done
> Marcelo !
>
> Thanks,
> Mridul
>
> On Fri, Jan 5, 2018 at 5:06 PM Marcelo Vanzin wrote:
>
>> Hey all, especially those working on the k8s stuff.
>>
>>
Agree it should be nice to have this simplification, and users can still
create their custom images by copy/modifying the default one.
Thanks for bring this out Marcelo!
2018-01-05 17:06 GMT-08:00 Marcelo Vanzin :
> Hey all, especially those working on the k8s stuff.
>
> Currently we have 3 docke
