Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

Just FYI, I had to come up with a better JQL to filter out the JIRAs that already have linked PRs. In case it helps someone, I use this JQL now to look through the open JIRAs: project = SPARK AND status = Open AND NOT issueFunction in linkedIssuesOfRemote("Github Pull Request *") ORDER BY created

Re: [Discuss] Follow ANSI SQL on table insertion

I have heard about many complaints about the old table insertion behavior. Blindly casting everything will leak the user mistake to a late stage of the data pipeline, and make it very hard to debug. When a user writes string values to an int column, it's probably a mistake and the columns are misor

[Discuss] Follow ANSI SQL on table insertion

Hi everyone, I would like to discuss the table insertion behavior of Spark. In the current data source V2, only UpCast is allowed for table insertion. I think following ANSI SQL is a better idea. For more information, please read the Discuss: Follow ANSI SQL on table insertion

Re: disable checkpointing in structured streaming

resurfacing this thread again.. On Mon, Jul 8, 2019 at 8:55 PM Charles vinodh wrote: > Hi , > > is it possible to disable checkpointing in structured streaming and have > it replaced by our own checkpointing implementation where the offsets are > saved in an external database?.. I looked up the