removing most of the config functions in SQLConf?

In SQLConf, for each config option, we declare them in two places: First in the SQLConf object, e.g.: *val* CSV_PARSER_COLUMN_PRUNING = buildConf ( "spark.sql.csv.parser.columnPruning.enabled" ) .internal() .doc( "If it is set to true, column names of the requested schema are passed to CSV pa

[build system] jenkins was in a mildly wedged state, just restarted

sorry if any pull request builds got killed, but jenkins was in a bad state and needed (another) full restart. looks like things are happily building now, and i can reliably log in through the UI. i'll keep an eye on things over the next couple of hours. shane -- Shane Knapp UC Berkeley EECS Re

Re: Self join

Thanks for the extra context, Marco. I thought you were trying to propose a solution. On Thu, Dec 13, 2018 at 2:45 AM Marco Gaido wrote: > Hi Ryan, > > My goal with this email thread is to discuss with the community if there > are better ideas (as I was told many other people tried to address th

Re: dsv2 remaining work

We discussed this issue in the sync. I'll be sending out a summary later today, but we came to a conclusion on some of these. For #1, there are 2 parts: the design and the implementation. We agreed that the design should not include SaveMode. The implementation may include SaveMode until we can re

RE: [SPARK-26160] Make assertNotBucketed call in DataFrameWriter::save optional

Great! Please add joaquin.guantergonzal...@telefonica.com to the list of attendees. Thanks, Ximo De: Ryan Blue Enviado el: lunes, 10 de diciembre de 2018 18:46 Para: JOAQUIN GUANTER GONZALBEZ CC: Wenchen Fan ; Spark Dev List Asunto: Re: [SPARK-

Re: Self join

Hi Ryan, My goal with this email thread is to discuss with the community if there are better ideas (as I was told many other people tried to address this). I'd consider this as a brainstorming email thread. Once we have a good proposal, then we can go ahead with a SPIP. Thanks, Marco Il giorno m