I am Ok with +1.
Having said that there is a merit IMO to add a matrix highlighting the
differences between real time and Continuous Processing (Continuous Mode)
to SPIP. Unless the assumption is that spark has abandoned the Continuous
Mode) altogether
*Feature Real-time Processing (via Micro-batching)
Continuous Processing (Continuous Mode)*
Underlying Model optimized micro-batching (small, frequent batches)
Truly continuous, event-at-a-time (or near event-at-a-time) processing via
long-running tasks.
Latency Goal Low latency (seconds to hundreds of milliseconds)
Ultra-low latency (single-digit or sub-millisecond) for true real-time.
for perceived real-time.
Fault Tolerance Highly robust, exactly-once processing guarantees
At-least-once processing is standard. Exactly-once for stateful operations
is more complex and depends on external systems (e.g., transactional
sinks) or careful custom
implementation for both stateless and stateful operations
etc
Dr Mich Talebzadeh,
Architect | Data Science | Financial Crime | Forensic Analysis | GDPR
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
On Sun, 1 Jun 2025 at 20:01, L. C. Hsieh <vii...@gmail.com> wrote:
> Hi all,
>
> I would like to start a vote on the new real-time mode in Apache Spark
> Structured Streaming.
>
> Discussion thread:
> https://lists.apache.org/thread/ovmfbzfkc3t9odvv5gs75fhpvdckn90f
> SPIP:
> https://docs.google.com/document/d/1CvJvtlTGP6TwQIT4kW6GFT1JbdziAYOBvt60ybb7Dw8/edit?tab=t.0#heading=h.ulas5788cm9t
> JIRA: https://issues.apache.org/jira/browse/SPARK-52330
>
> Please vote on the SPIP for the next 72 hours:
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don’t think this is a good idea because …
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>
