Re: Improved Structured Streaming Documentation Proof-of-Concept

I'm glad you think it's generally a good idea! I will mention, though, that with these better docs I've almost finished, I'm hoping that Structured Streaming no longer stays a specialist topic that requires "trench warfare." With good pedagogy, I think that it's very approachable. The Knowledge Sh

Re: Improved Structured Streaming Documentation Proof-of-Concept

Hi, Your intended work on improving the Structured Streaming documentation is great! Clear and well-organized instructions are important for everyone using Spark, beginners and experts alike. Having said that, Spark Structured Streaming much like other specialist topics with Spark say (k8s) or oth

Re: Allowing Unicode Whitespace in Lexer

While we're at it, maybe consider allowing "smart quotes" too :) -0xe1a On Sat, Mar 23, 2024 at 5:29 PM serge rielau.com wrote: > Hello, > > I have a PR https://github.com/apache/spark/pull/45620 ready to go that > will extend the definition of whitespace (what separates token) from the > smal

Improved Structured Streaming Documentation Proof-of-Concept

Hi all, I recently started an effort to improve the Structured Streaming documentation. I thought that the current documentation, while very comprehensive, could be improved in terms of organization, clarity, and presence of examples. You can view the repo here

Re: Vote on Dynamic resource allocation for structured streaming [SPARK-24815]

Hi Pavan, I looked at the PR, and the changes look simple and contained. It would be useful to add dynamic resource allocation to Spark Structured Streaming. Jungtaek. Would you be able to shepherd this change? On Tue, Mar 19, 2024 at 10:38 AM Bhuwan Sahni wrote: > Thanks a lot for creating t

Re: [DISCUSS] MySQL version support policy

Thanks Dongjoon’s reply and questions, > A. Adding a new Apache Spark community policy (contract) to guarantee MySQL > LTS Versions Support. Yes, at least the latest MySQL LTS version. To reduce the maintenance efforts on the Spark side, I think we can only run CI with the latest LTS version but