Re: Materialized view query rewriting with joins

Sorry, I missed Lattices as I didn't know about Lattices without aggregations. Btw there is this rule where we try to replace star table scan by Tiles of Lattices with filters: https://github.com/qubole/quark/blob/master/optimizer/src/main/java/com/qubole/quark/planner/FilterAggStarRule.java I thou

[jira] [Created] (CALCITE-1276) In Druid adapter, deduce columns by running a "segmentMetadata" query

Julian Hyde created CALCITE-1276: Summary: In Druid adapter, deduce columns by running a "segmentMetadata" query Key: CALCITE-1276 URL: https://issues.apache.org/jira/browse/CALCITE-1276 Project: Calc

Re: Checking compliance to a schema with Calcite

Hi Jacques, you are right in that JSON schema is the most natural solution for JSON. But I was looking to have a single SQL view of all the system. Maybe defining a conversion between SQL schemas and JSON schemas could be an approach that makes sense. I'll tell you guys if I'm able to make some pro

Re: Checking compliance to a schema with Calcite

Hi Julian, thanks for your answer. I was thinking on a method. I will tell you if I make progress in this line, first I have to get familiar with the Calcite API. Thanks again for your help. Greetings, Juan On Thu, Jun 2, 2016 at 1:17 PM, Julian Hyde wrote: > Would this be just a method, or

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

+1 Michael Mior wrote: I think this would be the ideal approach assuming there isn't too much overhead in maintaining different parsers and keeping them in sync. We wouldn't want to end up in the situation where the parsers start drifting and it could end up being a lot of work to maintain test

Re: Materialized view query rewriting with joins

Yes, I think you could make Lattices work without aggregation, as long as you keep the key assumption, which is that dropping joined tables does not change the number of rows (so, primary key and foreign key integrity, relationships are all many-to-one from the root (fact) table, and you have to

Re: Materialized view query rewriting with joins

Thanks for the pointer. Any chance there's a good example somewhere using lattices that would help me get something working? Also, is it possible to have lattices with no aggregation? Unfortunately either way it doesn't completely suit my use case since I'm going to have to do rewriting on joins as

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

I think this would be the ideal approach assuming there isn't too much overhead in maintaining different parsers and keeping them in sync. We wouldn't want to end up in the situation where the parsers start drifting and it could end up being a lot of work to maintain test cases for all the differen

Re: Calcite - Spark Adapter

You’re asking for a couple of things. First, a Spark adapter (able to generate code that uses Spark as an engine, and in particular use its implementations of the relational operators: join, aggregate etc.). Second, a Hive schema factory (able to read metadata from HCatalog). The first exists b

[jira] [Created] (CALCITE-1275) Add Hive schema factory

Julian Hyde created CALCITE-1275: Summary: Add Hive schema factory Key: CALCITE-1275 URL: https://issues.apache.org/jira/browse/CALCITE-1275 Project: Calcite Issue Type: Bug Repor

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

YAML isn’t an aside — thanks for raising it. Along with JSON and HOCON, YAML is another viable language for writing model files. I think any of these formats would plug into ModelHandler quite easily, because they all produce data in the JSON model. And performance isn’t important, because the

Re: Materialized view query rewriting with joins

Short answer: yes, if you use lattices. Remember, Calcite has two mechanisms for matching materialized views: the "general purpose" mechanism, and lattices. Using general purpose mechanism you can declare a materialized view based on any query, but there needs to be a unification rule to rewrit

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

Cool :) Justification enough for me. Jacques Nadeau wrote: Just to clarify, my responses were in regards to HOCON. On Mon, Jun 6, 2016 at 1:53 PM, Jacques Nadeau wrote: We use it a lot in Drill for configuration files. It serves this purpose well. (We especially leverage the caoncatenation

Re: Calicite Spark Adapter - java.lang.AssertionError: Expected: a string containing "PLAN=SparkToEnumerableConverter

You aren’t doing anything wrong. As a couple of people have noted recently on this list, the Spark adapter needs an update. I have logged a JIRA case: https://issues.apache.org/jira/browse/CALCITE-1274 . Any volunteers to work on it? Julian

[jira] [Created] (CALCITE-1274) Update Spark adapter

Julian Hyde created CALCITE-1274: Summary: Update Spark adapter Key: CALCITE-1274 URL: https://issues.apache.org/jira/browse/CALCITE-1274 Project: Calcite Issue Type: Bug Reporter

[VOTE] Release apache-calcite-1.8.0 (release candidate 0)

Hi all, I have created a build for Apache Calcite 1.8.0, release candidate 0. Thanks to everyone who has contributed to this release. It includes two new adapters (Druid and Elasticsearch), several important SQL enhancements, and a couple of dozen other features and bug-fixes. You can read the re

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

As an aside if we want a human-readable configuration format that's more flexible than JSON I would personally prefer YAML. It's much more standard than any of the JSON variants and can encode all the necessary structures. -- Michael Mior michael.m...@gmail.com 2016-06-06 16:51 GMT-04:00 Michael

Calicite Spark Adapter - java.lang.AssertionError: Expected: a string containing "PLAN=SparkToEnumerableConverter

Hi All, I am trying to evaluate Calcite spark adapter.I have the project setup locally on my machine. When running SparkAdapterTest I am getting below error. Any Idea what I am doing wrong and what should I do to resolve this issue ? java.lang.AssertionError:  Expected: a string containing "PLAN

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

Just to clarify, my responses were in regards to HOCON. On Mon, Jun 6, 2016 at 1:53 PM, Jacques Nadeau wrote: > We use it a lot in Drill for configuration files. It serves this purpose > well. (We especially leverage the caoncatenation/overlay capbilities.) > > I haven't thought enough about it

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

We use it a lot in Drill for configuration files. It serves this purpose well. (We especially leverage the caoncatenation/overlay capbilities.) I haven't thought enough about its use in this case to have an opinion one way or the other. On Mon, Jun 6, 2016 at 1:33 PM, Josh Elser wrote: > At a g

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

Right now none of the examples are strictly JSON anyway. There are several different JSON supersets that we could potentially use. It might be nice to pick one so there's at least some kind of standard. https://hjson.org/ https://github.com/timjansen/hanson http://json5.org/ -- Michael Mior micha

Re: Making Calcite release 1.8

Julian Hyde wrote: Now Avatica 1.8 is released, I’m building Calcite 1.8 RC0 now, and I’ll start a vote later today. I recently updated my public keys (and the KEYS file) and I might have screwed up, so please pay extra attention to the hashes this time. We completed most bugs and PRs. The

Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files

At a glance, it looks like it would be a nice improvement over JSON with the ability to still just provide json. Haven't had any practical experience with it though (e.g. does it work as advertised). Looks like they recommend ".conf" as the suffix The JSON superset is called "Human-Optimized C

Re: [ANNOUNCE] Apache Calcite Avatica 1.8.0 released

+1. Awesome job, Josh & the rest of the Avatica contributors! On Mon, Jun 6, 2016 at 12:53 PM, Julian Hyde wrote: > Thanks, Josh, for being release manager yet again. This release is a > significant improvement to Avatica on several fronts (security, performance > and quality). > > Julian > > >

Re: [ANNOUNCE] Apache Calcite Avatica 1.8.0 released

Thanks, Josh, for being release manager yet again. This release is a significant improvement to Avatica on several fronts (security, performance and quality). Julian > On Jun 6, 2016, at 8:54 AM, Josh Elser wrote: > > The Apache Calcite team is pleased to announce the release of Apache Calcit

[DISCUSS] Allow HOCON as a format for Calcite models and config files

HOCON is a superset of JSON that makes it easier for humans to edit JSON files. For example, it supports multi-line strings, file inclusion, comments. Should Calcite support HOCON in addition to strict JSON for model files any other config files? What should be the suffix of such files? https:/

Re: Several quotes symbols in sql parsing

Short answer: no. Long answer: We have an open development process. That means our “plans”, such as they are, are the open JIRA cases. Whether those features make it into a release depends on whether someone creates a PR for those cases, and whether that PR passes review. Do I personally think

Re: Making Calcite release 1.8

Now Avatica 1.8 is released, I’m building Calcite 1.8 RC0 now, and I’ll start a vote later today. I recently updated my public keys (and the KEYS file) and I might have screwed up, so please pay extra attention to the hashes this time. We completed most bugs and PRs. The following won’t make it

Re: Several quotes symbols in sql parsing

Julian, Are there any plans about the implementation of this improvement in the future calcite versions? Kind regards Vitalii 2016-06-02 18:03 GMT+00:00 Vitalii Diravka : > I see. Thanks for replying > > Kind regards > Vitalii > > 2016-06-01 19:25 GMT+00:00 Julian Hyde : > >> You can switch amon

Re: [RESULT] [VOTE] Apache Calcite Avatica 1.8.0 rc1

You're good to go. Sorry the entire cycle slipped a bit! Julian Hyde wrote: Thanks, Josh. I’m close to having an RC0 of Calcite 1.8.0 (depends on Avatica 1.8.0) and I’ll start a vote as soon as you officially announce Avatica 1.8.0. On Jun 4, 2016, at 1:32 PM, Josh Elser wrote: This vote p

[ANNOUNCE] Apache Calcite Avatica 1.8.0 released

The Apache Calcite team is pleased to announce the release of Apache Calcite Avatica 1.8.0. Avatica is a framework for building database drivers. Avatica defines a wire API and serialization mechanism for clients to communicate with a server as a proxy to a database. The reference Avatica clie