Equivalent of emptyDataFrame in StructuredStreaming

Hi, This is going to come off as a silly question with a stream being unbounded but this is problem that I have (created for myself). I am trying to build an ETL pipeline and I have a bunch of stages. val pipelineStages = List( new AddRowKeyStage(EvergreenSchema), new WriteToHBaseStage(hBase

Re: Continuous task retry support

> > *I found that task retries are currently not supported > > in > continuous processing mode. Is there an

Re: Plan on Structured Streaming in next major/minor release?

Can these things be added into this list? 1. [SPARK-24630] Support SQLStreaming in Spark This patch defines the Table API for StructStreaming 2. [SPARK-25937] Support user-defined schema in Kafka Source & Sink This patch make user easier to work with StructStreaming 3. SS supports dynam