[jira] [Created] (CALCITE-5011) CassandraAdapterDataTypesTest fails with initialization error
Alessandro Solimando created CALCITE-5011:
-
Summary: CassandraAdapterDataTypesTest fails with initialization
error
Key: CALCITE-5011
URL: https://issues.apache.org/jira/browse/CALCITE-5011
Project: Calcite
Issue Type: Bug
Components: tests
Affects Versions: 1.29.0
Reporter: Alessandro Solimando
Assignee: Alessandro Solimando
{noformat}
> Task :cassandra:test FAILED
FAILURE 0.0sec, org.apache.calcite.test.CassandraAdapterTest >
initializationError
com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException:
Cassandra timeout during SIMPLE write query at consistency LOCAL_ONE (1 replica
were required but only 0 acknowledged the write)
at
com.datastax.oss.driver.api.core.servererrors.WriteTimeoutException.copy(WriteTimeoutException.java:96)
at
com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53)
at
com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:30)
at
com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230)
at
com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:54)
at
com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:78)
at
org.cassandraunit.utils.CqlOperations.lambda$execute$0(CqlOperations.java:18)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at org.cassandraunit.CQLDataLoader.load(CQLDataLoader.java:34)
at
org.apache.calcite.test.CassandraAdapterTest.load(CassandraAdapterTest.java:52)
FAILURE 38.2sec,1 completed, 1 failed, 0 skipped,
org.apache.calcite.test.CassandraAdapterTest
{noformat}
The root cause is probably a session leak, as suggested by the warning:
{noformat}
CassandraAdapterDataTypesTest > testSimpleTypesRowType() STANDARD_OUT
2022-02-14 04:36:56,298 [ForkJoinPool-1-worker-1] WARN - You have too many
session instances: 15 active, expected less than 4 (see
'advanced.session-leak.threshold' in the configuration)
> Task :core:forbiddenApisMain
{noformat}
For the full error see:
https://ci.appveyor.com/project/ApacheSoftwareFoundation/calcite/builds/42562506/job/4ppus57wtv36689d
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
CoreRules not getting applied.
Hi devs, When trying to optimize the below query with calcite 1.29.0, it's not properly reducing the PLUS operation on date literal and interval literal. I'm using CoreRules.PROJECT_REDUCE_EXPRESSIONS select TIMESTAMPADD(DAY, 22, TIMESTAMP '1995-12-10 02:06:17') as constant_col from nation LogicalProject(constant_col=[+(1995-12-10 02:06:17, *(8640:INTERVAL DAY, 22))]) BindableTableScan(table=[[table-test-db, nation]], projects=[[]], aliases=[[constant_col]]) However, the same query is getting optimized to, calcite 1.23.0. LogicalProject(constant_col=[1996-01-01 02:06:17]) BindableTableScan(table=[[table-test-db, nation]], projects=[[]], aliases=[[constant_col]]) What could be the possible causes for this difference in the latest calcite version? Regards, Chathura
Re: Failed to import gradle
Thanks for the update!
> On Feb 14, 2022, at 11:46 PM, Aitozi wrote:
>
> Hi Julian&Gravin
>
>Sorry for the late response, I tried the solution posted by Gravin, The
> IDE can work with calcite project now. The changed files are
>
>modified: .idea/vcs.xml
>modified: build.gradle.kts
>modified: buildSrc/build.gradle.kts
>modified: core/build.gradle.kts
>modified: gradle.properties
>modified: release/build.gradle.kts
>
> I want to reproduce it by git stash and reopen the project, but it failed
> to reproduce.
>
> The error log before is :
>
> Unresolved reference: implementation :29
> Unresolved reference: implementation :30
> Unresolved reference: testimplementation:32
> Unresolved reference: testImplementation:33
> Unresolved reference: testimplementation:34
> Unresolved reference: testimplementation:35
> Unresolved reference: testimplementation:36
> Unresolved reference: testRuntime Only :38
> Unresolved reference: source Sets > Configure project :cassandra:68
> ...
>
> So now the problem is magically disappeared. If I can reproduce again, I
> will create a ticket for it.
>
> Thanks
> Aitozi
>
>
> Gavin Ray 于2022年2月9日周三 23:53写道：
>
>> Submitted a PR and Jira issue for this =)
>>
>> On Thu, Jan 27, 2022 at 6:44 PM Julian Hyde
>> wrote:
>>
>>> Can someone log a bug for this?
>>>
>>> I don’t run into it personally. (I just click open on the directory. I
>>> never invoke ‘import gradle’.) But people shouldn’t have to edit their
>>> Gradle files.
>>>
>>>
>>>
On Jan 27, 2022, at 6:55 AM, Gavin Ray wrote:
It didn't work for me either.
I had to remove some of the Gradle tasks, related to checks IIRC.
What I did that made it work was:
1. Modify "gradle.properties" in the root of the repo, adding the
>>> following
line:
skipAutostyle=true
2. Modify every instance of
"val skipAutostyle by props()"
to
"val skipAutostyle by props(true)"
in the Gradle build files
(This line exists in "build.gradle.kts" and
>> "buildSrc/build.gradle.kts")
3. Commented out the CheckStyle task in "core/build.gradle.kts"
//tasks.withType().configureEach {
//mustRunAfter(versionClass)
//mustRunAfter(javaCCMain)
//mustRunAfter(javaCCTest)
//}
4. Commented out "releaseArtifacts" plugin in
>> "release/build.gradle.kts"
//releaseArtifacts {
//artifact(archiveTask)
//}
My git status looks like:
user@MSI:~/projects/calcite$ git status
modified: build.gradle.kts
modified: buildSrc/build.gradle.kts
modified: core/build.gradle.kts
modified: gradle.properties
modified: release/build.gradle.kts
Hope this helps =)
On Thu, Jan 27, 2022 at 4:03 AM Thomas Rebele
>> >>>
wrote:
> Hello,
>
> the Calcite mailing list removes pictures. Could you please post the
>>> error
> message in text form?
>
> Cordialement / Best Regards,
> *Thomas Rebele, PhD* | R&D Developer | Germany | www.tibco.com
>
>
> On Thu, Jan 27, 2022 at 7:38 AM Aitozi wrote:
>
>> Hi community:
>> I'm new to calcite community and gradle. When I followed the
>>> Howto.md
>> to import the calcite project, It always failed with the following
>>> error.
>> After some search in Google and calcite dev mail list, I stiil can't
>>> not
>> figure it out. Is there anyone can help me out? thanks in advance
>>
>> [image: image.png]
>> The version of the intellij IDEA is 2021.1.3.
>>
>> Atiozi.
>>
>
>>>
>>>
>>
Re: [DISCUSS] Towards Calcite 1.30.0
+1 to release 1.30 soon, to keep up the tempo. Feb 28th sounds good to me. Thanks for pushing on this, Ruben. Andrei, Are you available to be RM for 1.30? When is a good date for you? Julian On Mon, Feb 14, 2022 at 9:16 AM Ruben Q L wrote: > > Hello, > > It has been almost two months since our last release [1], and I think we > should make an effort to continue keeping the rhythm of one release every > two months approximately. > > If I am not mistaken, the next release managers would be [2]: > - Andrei Sereda for 1.30.0 > - Liya Fan for 1.31.0 > > Are you guys still able to act as release managers for these versions? > Do we have any other volunteers for the subsequent releases? > > As usual, according to our Jira dashboard [3] and Github [4], there are > many pending issues that could / should be part of the release. What do you > think would be a reasonable date to propose for a RC? Perhaps 28th > February? I would like to let us have plenty of time to collectively review > the PR candidates and "unblock" (to some extent) the PR situation (see also > the recent separate thread on that matter [5]). > > Best regards, > Ruben > > > [1] https://calcite.apache.org/news/2021/12/25/release-1.29.0/ > [2] https://lists.apache.org/thread/kqslh8yqvxbolk74vov3ofw19h90xkp2 > [3] > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950 > [4] https://github.com/apache/calcite/pulls > [5] https://lists.apache.org/thread/30pf1o0vlcn7y3bhlcht1wdmvmxyvghn
Re: calcite multi-threading problem
Hi Stamatis，
Finally we are talking about the threading side about the problem,
can we open multiple connections from mutiple threads against the same
ReflectiveSchema schema with the same target object ?
-- Original --
From: "Stamatis Zampetakis";;
Send time: Monday, Feb 14, 2022 5:49 PM
To: "dev";
Subject: Re: calcite multi-threading problem
Hi,
I haven't gone through all the exchanges but wanted to mention that in
general the JDBC connection APIs are not thread-safe.
You previously mentioned the following:
"we open a calcite connection with ReflectiveSchema in one thread, and
execute sqls in other threads against the connection, but failed with
this error"
This is not a supported use case. Every thread should have its own
connection and you shouldn't use the same connection instance from multiple
threads.
Best,
Stamatis
On Sat, Feb 12, 2022 at 2:14 PM xiaobo wrote:
> 3.1.4 of janino and commons-compiler works now.
>
>
>
>
> -- Original --
> From: "xiaobo ";;
> Send time: Saturday, Feb 12, 2022 8:04 PM
> To: "dev";
>
> Subject: Re: calcite multi-threading problem
>
>
>
> but strange enough, the same code works in a single thread.
>
>
>
>
> -- Original --
> From: "xiaobo ";;
> Send time: Saturday, Feb 12, 2022 8:02 PM
> To: "dev";
>
> Subject: Re: calcite multi-threading problem
>
>
>
> we checked the janino release note page
> http://janino-compiler.github.io/janino/changelog.html
>
> Fixed issue #141: Unable to find org.codehaus.commons.compiler.properties
> in java 11: ICompilerFactories were loaded through the current thread's
> "context class loader", which is a bad choice in some environments. Thus,
> the methods "getDefaultCompilerFactory()", "getAllCompilerFactories()" and
> "getCompilerFactory(className)" were duplicated and augmented with a
> "classLoader" parameter.
>
> what does this mean? the getDefaultCompilerFactory method with a
> classloader parameter was removed intentionally?
> but it seems calcite still needs it , so what version of janino should we
> use for the latest calcite-core?
>
>
>
>
>
> -- Original --
> From: "xiaobo ";;
> Send time: Saturday, Feb 12, 2022 12:01 PM
> To: "dev";
>
> Subject: Re:calcite multi-threading problem
>
>
>
> In our use case, the Java class which opens calcite connection is
> contained in a external Java jar file, which is loaded dynamicly through
> classloaders other than the default systemloader like this:
>
>
> try {
> ClassLoader loader = URLClassLoader.newInstance(
> new URL[] { new URL("jar:file:" + path + "!/") },
> getClass().getClassLoader());
> Class clazz = Class.forName(classPath, true, loader);
>
> we mention this because the error message shows a classloader parameter.
>
>
>
>
> -- Original --
> From: "xiaobo ";;
> Send time: Saturday, Feb 12, 2022 11:56 AM
> To: "dev";
>
> Subject: Re: calcite multi-threading problem
>
>
>
> And if we remove calcite from our project, janino is not in "resolved
> depencies" library list of pom.xml,
> So we think it not the janino conflic problem.
>
>
>
>
> -- Original --
> From: "xiaobo ";;
> Send time: Saturday, Feb 12, 2022 9:49 AM
> To: "dev";
>
> Subject: Re: calcite multi-threading problem
>
>
>
> can you share the solution for the problem, we are using the following
> libraries, and it seems janino 3.1.6 is only dependent by calcite-core.
>
>
>
> UTF-8
> UTF-8
> 9
> 9
> 9
> 2.17.1
> 7.64.0.Final
>
>
> org.apache.calcite
> calcite-core
> 1.29.0
>
>
> org.apache.calcite.avatica
> avatica-core
> 1.20.0
>
>
>
>
> org.drools
> drools-core
> ${drools.version}
>
>
> org.drools
> drools-compiler
> ${drools.version}
>
>
> org.drools
> drools-decisiontables
> ${drools.version}
>
>
>
> -- Original --
> From: "xiong duan";;
> Send time: Saturday, Feb 12, 2022 8:43 AM
> To: "dev";
>
> Subject: Re: calcite multi-threading problem
>
>
>
> Yes, It is a dependency issue. I have met the same problem. +1 for Dmitry.
>
> Dmitry Sysolyatin 于2022年2月11日周五 21:18写道：
>
> > Actually, I had the same problem with spark. Spark 3.2.1 uses the old
> > version of janino, calcite 1.29 uses the new version of janino and they
> are
> > not compatible.
> > If I downgrade janino version to version which spark uses I got an error
> > not method found exception
> > If I upgrade the janino version to the version which calcite uses I get
> the
> > same error but with another method.
> >
> > As a resu
Re: [DISCUSS] Towards Calcite 1.30.0
+1 to release 1.30. Feb 28th sounds good to me. Julian Hyde 于2022年2月16日周三 08:27写道： > +1 to release 1.30 soon, to keep up the tempo. Feb 28th sounds good to > me. Thanks for pushing on this, Ruben. > > Andrei, Are you available to be RM for 1.30? When is a good date for you? > > Julian > > On Mon, Feb 14, 2022 at 9:16 AM Ruben Q L wrote: > > > > Hello, > > > > It has been almost two months since our last release [1], and I think we > > should make an effort to continue keeping the rhythm of one release every > > two months approximately. > > > > If I am not mistaken, the next release managers would be [2]: > > - Andrei Sereda for 1.30.0 > > - Liya Fan for 1.31.0 > > > > Are you guys still able to act as release managers for these versions? > > Do we have any other volunteers for the subsequent releases? > > > > As usual, according to our Jira dashboard [3] and Github [4], there are > > many pending issues that could / should be part of the release. What do > you > > think would be a reasonable date to propose for a RC? Perhaps 28th > > February? I would like to let us have plenty of time to collectively > review > > the PR candidates and "unblock" (to some extent) the PR situation (see > also > > the recent separate thread on that matter [5]). > > > > Best regards, > > Ruben > > > > > > [1] https://calcite.apache.org/news/2021/12/25/release-1.29.0/ > > [2] https://lists.apache.org/thread/kqslh8yqvxbolk74vov3ofw19h90xkp2 > > [3] > > > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12333950 > > [4] https://github.com/apache/calcite/pulls > > [5] https://lists.apache.org/thread/30pf1o0vlcn7y3bhlcht1wdmvmxyvghn >
Re: Allow Cascades driver invoking "derive" on the nodes produced by "passThrough"
> So the idea of keeping even internal components of Calcite "more public" is > rather a good thing than the bad one from my point of view. This can go two ways. There is the type of change that I call “drill a hole” where someone requests that an implementation detail be made public so that they can use it, but they don’t provide much justification, and don’t provide a use case that might be useful to others. The other is where someone converts the internal mechanism into a genuine feature, with a use case, documentation, and comprehensive tests that help others understand and use the feature. The “drill a hole” changes are a net burden on the Calcite project because we have to maintain the implementation going forward. I’m not going to make a judgment about whether this proposed change is positive. The key thing is to think of it as adding a feature, not just adding the word “public” in one or two source files. Julian
[jira] [Created] (CALCITE-5012) SubstitutionVisitor need a new rule to support remain top calc in mv.
Xurenhe created CALCITE-5012:
Summary: SubstitutionVisitor need a new rule to support remain top
calc in mv.
Key: CALCITE-5012
URL: https://issues.apache.org/jira/browse/CALCITE-5012
Project: Calcite
Issue Type: Bug
Reporter: Xurenhe
SubstitutionVisitor is a bottom-up to rewrite query's plan by mv.
Rewriting query successfully, when mv's plan has become a part of the query
plan.
The current rules are as follows:
{code:java}
AggregateOnCalcToAggregateUnifyRule
AggregateToAggregateUnifyRule
CalcToCalcUnifyRule
IntersectOnCalcsToIntersectUnifyRule
IntersectToIntersectUnifyRule
JoinOnCalcsToJoinUnifyRule
JoinOnLeftCalcToJoinUnifyRule
JoinOnRightCalcToJoinUnifyRule
ScanToCalcUnifyRule
TrivialRule
UnionOnCalcsToUnionUnifyRule
UnionToUnionUnifyRule {code}
Their aims:
* Predicate compensation by pulling-up
* Project compensation by pulling-up
* Aggregate roll-up
But, Rewriting query fails, when top rel of mv is calc.
And this rel just works for rearrangement of field's order or unused value
filling.
Let me give some examples:
Case1
{code:java}
--query
SELECT deptno,
sum(salary),
sum(k)
FROM
(SELECT deptno,
salary,
100 AS k
FROM emps)
GROUP BY deptno;
--mv
SELECT deptno,
sum(salary),
sum(commission) + 1,
sum(k)
FROM
(SELECT deptno,
salary,
commission,
100 AS k
FROM emps)
GROUP BY deptno;
--query's plan
LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)], EXPR$2=[SUM($2)])
LogicalCalc(expr#0..4=[{inputs}], expr#5=[100], deptno=[$t1], salary=[$t3],
k=[$t5])
LogicalTableScan(table=[[hr, emps]])
--mv's plan
LogicalCalc(expr#0..3=[{inputs}], expr#4=[1], expr#5=[+($t2, $t4)],
proj#0..1=[{exprs}], EXPR$2=[$t5], EXPR$3=[$t3])
LogicalAggregate(group=[{0}], EXPR$1=[SUM($1)], agg#1=[SUM($2)],
EXPR$3=[SUM($3)])
LogicalCalc(expr#0..4=[{inputs}], expr#5=[100], deptno=[$t1], salary=[$t3],
commission=[$t4], k=[$t5])
LogicalTableScan(table=[[hr, emps]]) {code}
Case2
{code:java}
--query
SELECT deptno,
salary
FROM emps
UNION
SELECT deptno,
salary
FROM emps;
--mv
SELECT deptno,
salary,
'hello' AS k
FROM
(SELECT deptno,
salary
FROM emps
UNION SELECT deptno,
salary
FROM emps);
--query's plan
LogicalUnion(all=[false])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
--mv's plan
LogicalCalc(expr#0..1=[{inputs}], expr#2=['hello'], proj#0..2=[{exprs}])
LogicalUnion(all=[false])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]])
LogicalCalc(expr#0..4=[{inputs}], deptno=[$t1], salary=[$t3])
LogicalTableScan(table=[[hr, emps]]) {code}
We could define a new rule of HolderToCalcUnifyRule to solve these problems
Related issues:
* CALCITE-5000: https://issues.apache.org/jira/browse/CALCITE-5000
* CALCITE-4376: https://issues.apache.org/jira/browse/CALCITE-4376
Holder is top wrapped rel for query, link to
`org.apache.calcite.rel.mutable.Holder`
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
questions about materialization view with windowing function
hello, I have a question about the materialization view with windowing function. Like the following，can the unifyRule and the MaterializedViewRule match？ My understanding is that， materialization view with windowing function must not exist filter and select items must can be expressed from view select item expressions the unifyRule focus on part pattern，so it can't match. the MaterializedViewRule can't match too, I can't find any code adout that sofa. but if I want to match, I should extend it in MaterializedViewRule but not unifyRule I really want to know that Is my understanding right ? materialization view with windowing function case: SELECT ds,workspace_name, sum(1) over(PARTITIONby ds,workspace_name orderby ds asc) as sum_cnt, sum(1) over(PARTITIONby ds orderby ds asc) as sum_cnt from dwd_airengine_query_pattern_delta_ht ; I look forward to hearing from you very much thanks rules transformation Permalink
