Was there any answer to my question around the effect of changes to the sink api regarding access to underlying offsets?
On Wed, Nov 1, 2017 at 11:32 AM, Reynold Xin <r...@databricks.com> wrote: > Most of those should be answered by the attached design sketch in the JIRA > ticket. > > On Wed, Nov 1, 2017 at 5:29 PM Debasish Das <debasish.da...@gmail.com> > wrote: >> >> +1 >> >> Is there any design doc related to API/internal changes ? Will CP be the >> default in structured streaming or it's a mode in conjunction with exisiting >> behavior. >> >> Thanks. >> Deb >> >> On Nov 1, 2017 8:37 AM, "Reynold Xin" <r...@databricks.com> wrote: >> >> Earlier I sent out a discussion thread for CP in Structured Streaming: >> >> https://issues.apache.org/jira/browse/SPARK-20928 >> >> It is meant to be a very small, surgical change to Structured Streaming to >> enable ultra-low latency. This is great timing because we are also designing >> and implementing data source API v2. If designed properly, we can have the >> same data source API working for both streaming and batch. >> >> >> Following the SPIP process, I'm putting this SPIP up for a vote. >> >> +1: Let's go ahead and design / implement the SPIP. >> +0: Don't really care. >> -1: I do not think this is a good idea for the following reasons. >> >> >> > --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
