roduce invalid results. My intuition is yes, because
>>> different users have different levels of tolerance for different kinds of
>>> errors. I’d expect these sorts of configurations to be set up at an
>>> infrastructure level, e.g. to maintain consistent standards throughout
t;> situations that could produce invalid results. My intuition is yes, because
>>> different users have different levels of tolerance for different kinds of
>>> errors. I’d expect these sorts of configurations to be set up at an
>>> infrastructure level, e.g. to maintain co
le organization.
>>
>>
>>
>> *From: *Gengliang Wang
>> *Date: *Thursday, August 1, 2019 at 3:07 AM
>> *To: *Marco Gaido
>> *Cc: *Wenchen Fan , Hyukjin Kwon <
>> gurwls...@gmail.com>, Russell Spitzer , Ryan
>> Blue , Reynold Xin , Matt Che
m: *Gengliang Wang
> *Date: *Thursday, August 1, 2019 at 3:07 AM
> *To: *Marco Gaido
> *Cc: *Wenchen Fan , Hyukjin Kwon ,
> Russell Spitzer , Ryan Blue ,
> Reynold Xin , Matt Cheah ,
> Takeshi Yamamuro , Spark dev list <
> dev@spark.apache.org>
> *Subject: *Re: [Discuss] Fo
ido
Cc: Wenchen Fan , Hyukjin Kwon ,
Russell Spitzer , Ryan Blue ,
Reynold Xin , Matt Cheah , Takeshi
Yamamuro , Spark dev list
Subject: Re: [Discuss] Follow ANSI SQL on table insertion
Hi all,
Let me explain a little bit on the proposal.
By default, we follow the store assignmen
arranted to do so.
>>
>>
>>
>> -Matt Cheah
>>
>>
>>
>> *From: *Reynold Xin
>> *Date: *Wednesday, July 31, 2019 at 9:58 AM
>> *To: *Matt Cheah
>> *Cc: *Russell Spitzer , Takeshi Yamamuro <
>> linguin@gmail.com>, Ge
w...@databricks.com >, Ryan Blue <
> >rb...@netflix.com
> >, Spark dev list < dev@spark.apache.org >, Hyukjin Kwon < gurwls...@gmail.com
> >, Wenchen Fan < cloud0...@gmail.com >
> *Subject:* Re: [Discuss] Follow ANSI SQL on table insertion
>
>
>
Date: Wednesday, July 31, 2019 at 9:58 AM
To: Matt Cheah
Cc: Russell Spitzer , Takeshi Yamamuro
, Gengliang Wang , Ryan
Blue , Spark dev list , Hyukjin Kwon
, Wenchen Fan
Subject: Re: [Discuss] Follow ANSI SQL on table insertion
Matt what do you mean by maximizing 3, while allowing not
n Fan < cloud0...@gmail.com >
> *Cc:* Russell Spitzer < russell.spit...@gmail.com >, Takeshi Yamamuro <
> linguin@gmail.com
> >, Gengliang Wang < gengliang.w...@databricks.com >, Ryan Blue <
> >rb...@netflix.com
> >, Spark dev list < dev@
perhaps the behavior can be flagged
by the destination writer at write time.
-Matt Cheah
From: Hyukjin Kwon
Date: Monday, July 29, 2019 at 11:33 PM
To: Wenchen Fan
Cc: Russell Spitzer , Takeshi Yamamuro
, Gengliang Wang , Ryan
Blue , Spark dev list
Subject: Re: [Discuss] Follow ANSI SQL
t; catch this earlier during analysis than to run most of a job and then
>>>>>>>> fail.
>>>>>>>>
>>>>>>>> In addition, part of the justification for using the ANSI standard
>>>>>>>> is to avoid b
ting
>>>>>>> validation rules that require explicit casts to truncate values.
>>>>>>>
>>>>>>> That still leaves the use case that motivated this proposal, which
>>>>>>> is that floating
>>>> On Thu, Jul 25, 2019 at 9:11 AM Wenchen Fan
>>>>>> wrote:
>>>>>>
>>>>>>> I have heard about many complaints about the old table insertion
>>>>>>> behavior. Blindly casting everything will leak the user m
>>>>> columns are misordered in the INSERT statement. We should fail the query
>>>>>> earlier and ask users to fix the mistake.
>>>>>>
>>>>>> In the meanwhile, I agree that the new table insertion behavi
table insertion behavior we
>>>>> introduced for Data Source V2 is too strict. It may fail valid queries
>>>>> unexpectedly.
>>>>>
>>>>> In general, I support the direction of following the ANSI SQL
>>>>> standard. But
t;> Gaido is working on it already. This will have a config as well and by
>>>> default we still return null.
>>>>
>>>> After doing this, the default behavior will be slightly different from
>>>> the SQL standard (cast can return null), and user
t;> After doing this, the default behavior will be slightly different from
>>> the SQL standard (cast can return null), and users can turn on the ANSI
>>> mode to fully follow the SQL standard. This is much better than before and
>>> should prevent a lot of user mistake
>> should prevent a lot of user mistakes. It's also a reasonable choice to me
>> to not throw exceptions at runtime by default, as it's usually bad for
>> long-running jobs.
>>
>> Thanks,
>> Wenchen
>>
>> On Thu, Jul 25, 2019 at 11:37 PM
On Thu, Jul 25, 2019 at 11:37 PM Gengliang Wang <
> gengliang.w...@databricks.com> wrote:
>
>> Hi everyone,
>>
>> I would like to discuss the table insertion behavior of Spark. In the
>> current data source V2, only UpCast is allowed for table insertion. I think
>
> wrote:
> Hi everyone,
>
> I would like to discuss the table insertion behavior of Spark. In the
> current data source V2, only UpCast is allowed for table insertion. I think
> following ANSI SQL is a better idea.
> For more information, please read the Discuss: F
Hi everyone,
I would like to discuss the table insertion behavior of Spark. In the
current data source V2, only UpCast is allowed for table insertion. I think
following ANSI SQL is a better idea.
For more information, please read the Discuss: Follow ANSI SQL on table
insertion
<ht
21 matches
Mail list logo
