Re: Spark on Kubernetes Builder Pattern Design Document

2018-02-05 Thread Mark Hamstra
ork and migrate all the work done on the fork into the main > line. > > > > -Matt Cheah > > > > *From: *Mark Hamstra > *Date: *Monday, February 5, 2018 at 1:44 PM > *To: *Matt Cheah > *Cc: *"dev@spark.apache.org" , " > ramanath...@googl

Re: Spark on Kubernetes Builder Pattern Design Document

2018-02-05 Thread Matt Cheah
e.org" , "ramanath...@google.com" , Ilan Filonenko , Erik , Marcelo Vanzin Subject: Re: Spark on Kubernetes Builder Pattern Design Document That's good, but you should probably stop and consider whether the discussions that led up to this document's creation could hav

Re: Spark on Kubernetes Builder Pattern Design Document

2018-02-05 Thread Mark Hamstra
That's good, but you should probably stop and consider whether the discussions that led up to this document's creation could have taken place on this dev list -- because if they could have, then they probably should have as part of the whole spark-on-k8s project becoming part of mainline spark deve

Spark on Kubernetes Builder Pattern Design Document

2018-02-05 Thread Matt Cheah
Hi everyone, While we were building the Spark on Kubernetes integration, we realized that some of the abstractions we introduced for building the driver application in spark-submit, and building executor pods in the scheduler backend, could be improved for better readability and clarity. We