Re: Non deterministic result with Table API SQL

2018-11-05 Thread Fabian Hueske
Thanks Flavio for reporting the error helping to debug it. A job to reproduce the error is very valuable :-) Best, Fabian Am Mo., 5. Nov. 2018 um 14:38 Uhr schrieb Flavio Pompermaier < pomperma...@okkam.it>: > Here it is the JIRA ticket and, attached to if, the Flink (Java) job to > reproduce th

Re: Non deterministic result with Table API SQL

2018-11-05 Thread Flavio Pompermaier
Here it is the JIRA ticket and, attached to if, the Flink (Java) job to reproduce the error: https://issues.apache.org/jira/browse/FLINK-10795 On Wed, Oct 31, 2018 at 4:46 PM Timo Walther wrote: > As far as I know STDDEV_POP is translated into basic aggregate functions > (SUM/AVG/COUNT). But if

Re: Non deterministic result with Table API SQL

2018-10-31 Thread Timo Walther
As far as I know STDDEV_POP is translated into basic aggregate functions (SUM/AVG/COUNT). But if this error is reproducible in a little test case, we should definitely track this in JIRA. Am 31.10.18 um 16:43 schrieb Flavio Pompermaier: Adding more rows to the dataset lead to a deterministic e

Re: Non deterministic result with Table API SQL

2018-10-31 Thread Flavio Pompermaier
Adding more rows to the dataset lead to a deterministic error. My tests says that the problem arise when adding the STDDEV_POP to the query.. Do you think it could be possible that there's a concurrency problem in its implementation?

Re: Non deterministic result with Table API SQL

2018-10-31 Thread Flavio Pompermaier
I read a Parquet file from the filesystem. The input rows are always read in the same way, but results are different. My query is very big and maybe this affects somehow the query execution: UM(CASE WHEN isComplete(nome,sesso,cfPiva,cognome,immobili,deceduto,dataNascita,luogoNascita,impostePagate,

Re: Non deterministic result with Table API SQL

2018-10-31 Thread Timo Walther
Hi Flavio, do you execute this query in a batch or stream execution environment? In any case this sounds very strange to me. But is it guarateed that it is not the fault of the connector? Regars, Timo Am 31.10.18 um 14:54 schrieb Flavio Pompermaier: Hi to all, I'm using Flink 1.6.1 and I g

Non deterministic result with Table API SQL

2018-10-31 Thread Flavio Pompermaier
Hi to all, I'm using Flink 1.6.1 and I get different results when running the same query on the same static dataset. There are times that I get a 'NaN' as result of a select field-expression, while other times I get a valid double. How is this possible? This seems to happen only when I execute a co