[jira] [Created] (CALCITE-6873) The FilterProjectTransposeRule should not push the Filter past the Project when the Filter contains a Subquery with correlation

2025-03-04 Thread xiong duan (Jira)
xiong duan created CALCITE-6873: --- Summary: The FilterProjectTransposeRule should not push the Filter past the Project when the Filter contains a Subquery with correlation Key: CALCITE-6873 URL: https://issues.apache

[ANNOUNCE] Apache Calcite Avatica Go 5.4.0 released

2025-03-04 Thread Francis Chuang
The Apache Calcite team is pleased to announce the release of Apache Calcite Avatica Go 5.4.0. Avatica is a framework for building database drivers. Avatica defines a wire API and serialization mechanism for clients to communicate with a server as a proxy to a database. The reference Avatica c

Re: [DISCUSS] Towards Calcite 1.39.0

2025-03-04 Thread Xiong Duan
+1 for the first RC on March 10, 2025. Julian Hyde 于2025年3月5日周三 03:40写道: > > +1 for March 10th RC > > There are only 4 open issues tagged 1.39 [ > https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20fixVersion%20%3D%201.39.0%20%20AND%20%20%20statusCategory%20%20!%3D%20Don

Re: RelFieldTrimmer not optimally trimming after filters under joins?

2025-03-04 Thread Steven Phillips
Yes, whether it's more efficient or not depends on engine. For columnar engines, dropping the column is trivial and would be unambiguously better. Another reason it's better to use rules and let the optimizer figure out what's best. On Tue, Mar 4, 2025 at 4:42 PM Julian Hyde wrote: > I see. RelF

Re: RelFieldTrimmer not optimally trimming after filters under joins?

2025-03-04 Thread Julian Hyde
I see. RelFieldTrimmer could insert a Project right after the Filter, but in most calling conventions that plan would probably be less efficient. The field will be removed next time there is a Project or Aggregate. > On Mar 4, 2025, at 4:33 PM, Steven Phillips wrote: > > Julian, > The input co

Re: RelFieldTrimmer not optimally trimming after filters under joins?

2025-03-04 Thread Steven Phillips
Julian, The input column $1 is needed for the filter condition on the node below the join, and not needed for anything else above that. The join tells the filter below that it doesn't need that field. But the filter itself does not the field. And filters don't have the ability to remove fields (i.e

Re: RelFieldTrimmer not optimally trimming after filters under joins?

2025-03-04 Thread Julian Hyde
I don’t think I understand this conversation. RelFieldTrimmer is intended to be invoked on the whole tree. Each node, when invoking the trimmer on its input (child), tells the trimmer which of the fields of that input it actually uses. Now ‘which fields it actually uses’ is based on the fields t

Re: RelFieldTrimmer not optimally trimming after filters under joins?

2025-03-04 Thread Ian Bertolacci
I just hacked together an override where it will build a redundant project on each side if necessary. That should eliminate any overhead of invoking any planners or rules. (For our needs, additional projects have not performance implications) -Ian From: Ian Bertolacci Reply-To: "dev@calcite.apac

Re: RelFieldTrimmer not optimally trimming after filters under joins?

2025-03-04 Thread Ian Bertolacci
> I think you could work around this by always inserting trivial projects over > every node in the tree before trimming, and then clean up with > ProjectRemoveRule. This is pretty much exactly what I was doing. Good to know that I’m not wildly off-track Thanks! -Ian From: Steven Phillips Re

Re: RelFieldTrimmer not optimally trimming after filters under joins?

2025-03-04 Thread Steven Phillips
In think this is a current limitation of FieldTrimmer. The Join and Filter nodes can't drop columns (since they don't carry column selection information), and the trimmer doesn't add Project nodes (currently). I have worked around this limitation by using HepPlanner with various ProjectTranspose ru

RelFieldTrimmer not optimally trimming after filters under joins?

2025-03-04 Thread Ian Bertolacci
I’m looking at using RelFieldTrimmer, and I’m noticing that if a side of a join has unnecessary fields after a filter, there is no trim-fields project on that side to reduce the width of the row. Is this expected, or is there a configuration or pre-processing step that I am missing? For example

Re: Question about scale after parsing day/time intervals to RexLiteral

2025-03-04 Thread Mark Lewis
Hi Julian, Thank you for your reply. I understand now that "parsing" is the wrong term. What is the correct term I should be using? "Converting"? I am not sure that the user has any control of this behaviour using the type system. Calcite is hard-coded to represent intervals in millisecond prec

[RESULT] [VOTE] Release apache-calcite-avatica-go-5.4.0 (release candidate 0)

2025-03-04 Thread Francis Chuang
Thanks to everyone who has tested the release candidate and given their comments and votes. The tally is as follows. 3 binding +1s: Francis Chuang Ruben Quesada Lopez Stamatis Zampetakis 1 non-binding +1s: Alessandro Solimando No 0s or -1s. Therefore I am delighted to announce that the propos

Re: [DISCUSS] Towards Calcite 1.39.0

2025-03-04 Thread Julian Hyde
+1 for March 10th RC There are only 4 open issues tagged 1.39 [ https://issues.apache.org/jira/issues/?jql=project%20%3D%20CALCITE%20AND%20fixVersion%20%3D%201.39.0%20%20AND%20%20%20statusCategory%20%20!%3D%20Done ] > On Mar 4, 2025, at 2:56 AM, Alessandro Solimando > wrote: > > I share the

Re: Question about scale after parsing day/time intervals to RexLiteral

2025-03-04 Thread Julian Hyde
Sorry to be pedantic but it’s important. To me, ‘parse’ means to go from text to an AST, without applying judgments such as type system. As far as I can tell, there is no problem with ‘parsing’. The parser produces an AST, specifically a SqlLiteral, with all of the digits that were in the SQL te

Re: [VOTE] Release apache-calcite-avatica-go-5.4.0 (release candidate 0)

2025-03-04 Thread Alessandro Solimando
+1 (non-binding) after verifying as follows: - verified gpg signature: OK [1] - verified package checksum: OK [2] - running tests from sources: OK [3] - running tests on commit: OK [4] - checked release notes: OK - checking difference in folder: OK [5] Tested on Ubuntu 24.04.2 LTS x86_64. T

Re: SQL Validation Issue - Need help

2025-03-04 Thread Ruben Q L
Hello Dipti, You mention that "the same query works on some of the machines and is impacting few other", could you provide more context? In which environment(s) does it work? In which doesn't? In the environment(s) where it fails, is this failure systematic or somewhat "random" (do you have differe

Re: Question about scale after parsing day/time intervals to RexLiteral

2025-03-04 Thread Mark Lewis
Hi Mihai, Thank you for the reply and the pointers to those existing issues. I think the test failures I see when attempting different approaches to fixing the behaviour are a combination of bad tests — tests that just test the output of the current implementation, even if this is incorrect — a

Re: [DISCUSS] Towards Calcite 1.39.0

2025-03-04 Thread Alessandro Solimando
I share the same feeling, there are no blockers currently and the current release is already pretty massive. Thanks Stamatis for this hard work! On Tue, 4 Mar 2025 at 11:54, Francis Chuang wrote: > I think this is a reasonable plan as this release is quite behind. The > nice to haves, unless the

Re: [DISCUSS] Towards Calcite 1.39.0

2025-03-04 Thread Francis Chuang
I think this is a reasonable plan as this release is quite behind. The nice to haves, unless they can be committed by March 10, can go into the next release (approx. 3 months away) if we get back to our usual release cadence. On 4/03/2025 9:46 pm, Stamatis Zampetakis wrote: Hi all, The 1.39.

Re: [DISCUSS] Towards Calcite 1.39.0

2025-03-04 Thread Stamatis Zampetakis
Hi all, The 1.39.0 release already has more than 200 tickets and given that we are behind our usual (~3 months) release cadence I would suggest cutting an RC rather soon. I went over the tickets in the 1.39.0 release dashboard [1] and didn't see any release blockers. I removed the fix version for

[jira] [Created] (CALCITE-6872) Release Calcite 1.39.0

2025-03-04 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created CALCITE-6872: Summary: Release Calcite 1.39.0 Key: CALCITE-6872 URL: https://issues.apache.org/jira/browse/CALCITE-6872 Project: Calcite Issue Type: Task

[jira] [Created] (CALCITE-6871) Upgrade avatica from 1.25.0 to 1.26.0

2025-03-04 Thread Stamatis Zampetakis (Jira)
Stamatis Zampetakis created CALCITE-6871: Summary: Upgrade avatica from 1.25.0 to 1.26.0 Key: CALCITE-6871 URL: https://issues.apache.org/jira/browse/CALCITE-6871 Project: Calcite Iss

Re: [VOTE] Release apache-calcite-avatica-go-5.4.0 (release candidate 0)

2025-03-04 Thread Stamatis Zampetakis
Ubuntu 22.04.5 LTS, Docker version 27.2.0, Docker Compose version v2.20.3 * Checked signatures and checksums OK * Checked diff between repo and artifacts (diff -qr src git) OK * Went over release note OK * Checked README, NOTICE, LICENSE OK * All source files have ASF headers (grep -RiL "Lice