Re: Spark Structured Streaming and Flask REST API for Real-Time Data Ingestion and Analytics.

Hi Ashok, Thanks for pointing out the databricks article Scalable Spark Structured Streaming for REST API Destinations | Databricks Blog I browsed it and it is basically similar to many of us involved with

Re: Spark Structured Streaming and Flask REST API for Real-Time Data Ingestion and Analytics.

Please also note that Flask, by default, is a single-threaded web framework. While it is suitable for development and small-scale applications, it may not handle concurrent requests efficiently in a production environment. In production, one can utilise Gunicorn (Green Unicorn) which is a WSGI ( We

Re: [Spark Structured Streaming on K8S]: Debug - File handles/descriptor (unix pipe) leaking

Hello Dev! Spark structured streaming job with simple window aggregation is leaking file descriptor on kubernetes as cluster manager setup. It seems bug. I am suing HDFS as FS for checkpointing. Have anyone observed same? Thanks for any help. Please find more details in trailing email. For more

Re: Spark structured streaming

> > > - > > "Courage doesn't always roar. Sometimes courage is the quiet voice at the > > end of the day saying I will try again" > > > > > > > >

Re: Spark structured streaming

wski > To:Praveen Devarao/India/IBM@IBMIN > Cc: user , dev > Date:08/03/2016 04:17 pm > Subject:Re: Spark structured streaming > > > > > Hi Praveen, > > I've spent few hours on the changes rel

Re: Spark structured streaming

try again" From: Jacek Laskowski To: Praveen Devarao/India/IBM@IBMIN Cc: user , dev Date: 08/03/2016 04:17 pm Subject: Re: Spark structured streaming Hi Praveen, I've spent few hours on the changes related to streaming dataframes (included in the SPARK-8360) and

Re: Spark structured streaming

Hi Praveen, I've spent few hours on the changes related to streaming dataframes (included in the SPARK-8360) and concluded that it's currently only possible to read.stream(), but not write.stream() since there are no streaming Sinks yet. Pozdrawiam, Jacek Laskowski https://medium.com/@jacekl