Exactly, ST_LINE and ST_POINT are the dynamic type of an object with a GEOMETRY/GEOGRAPHY static type. The VARIANT is similar.
The VARIANT implementation is a proof that this can be done within the framework. Mihai ________________________________ From: David Chaava via dev <[email protected]> Sent: Friday, November 6, 2026 5:02 AM To: [email protected] <[email protected]> Cc: David Chaava <[email protected]> Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite Hi Mihai, Thanks for taking a look. Let me make the proposed typing rules more concrete. Suppose `places` has a nullable column named `g` of SQL type `GEOGRAPHY`, and the spatial operator table is enabled. Under the proposal, the following queries would behave as shown. SELECT g FROM places -- The result column has type GEOGRAPHY. SELECT COALESCE(g, NULL) FROM places -- The result column has type GEOGRAPHY and is nullable. SELECT ST_Area(g) FROM places -- Validation fails because the existing ST_Area expects GEOMETRY. At validation time, Calcite knows from the schema that `g` has SQL type `GEOGRAPHY`. It uses that type to determine the result type of the first two queries and to check whether `g` matches the declared `ST_Area` signature in the third. These checks happen before Calcite reads any rows. A point, a line, and a polygon are different shapes that a `GEOGRAPHY` value can represent, and all three have the same SQL type in this design. By comparison, a `VARIANT` value carries the SQL type of its contents at runtime. We cited CALCITE-4918 as a precedent for separating parser and validator work from runtime work. If you had a particular expression or operation in mind when drawing the analogy with `VARIANT`, we'd be happy to walk through how the proposed typing rules would apply to it. I hope this helps clarify the design. If you have any other questions, we'd be happy to walk through more examples. Best, David On Fri, Nov 6, 2026 at 12:31 AM Mihai Budiu <[email protected]> wrote: > I looked, but I am not much more enlightened. > > Maybe GEOMETRY/GEOGRAPHY will work a little like specialized forms > VARIANT, which is really dynamically-typed. > > Mihai > > ------------------------------ > *From:* David Chaava via dev <[email protected]> > *Sent:* Thursday, November 5, 2026 3:45 AM > *To:* [email protected] <[email protected]> > *Cc:* David Chaava <[email protected]> > *Subject:* Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > Hi Mihai, > > Thanks for raising this. We have a design proposal available here: > > https://docs.google.com/document/d/1Btqd0oZT8ctqIaGI_AROyAFJQe68RsVyeQbtmK4wHQw/edit > > We would appreciate your feedback on it before implementation. > > The proposal introduces GEOGRAPHY as a statically known SQL and relational > type. A column would be declared as GEOGRAPHY by its schema or adapter, and > Calcite would preserve that type through validation and planning. Point, > line, and polygon values would all have the same SQL type in this initial > design. > > The runtime dispatch mentioned earlier was a possibility for spatial > functions explicitly supporting both GEOMETRY and GEOGRAPHY. It would still > require validation of the declared argument types. The first contribution > focuses on the type itself and its propagation through a plan. > > The proposal includes assignment and coercion rules, nullability, nested > types, and the intended behavior of comparisons and set operations. > Equality and grouping are among the areas where we are seeking feedback on > the guarantees Calcite needs. > > Could you help us understand which aspects you have in mind when referring > to dynamic typing? We would appreciate your guidance on where the proposed > typing rules need further clarification. > > Best, > David > > On Wed, Nov 4, 2026 at 8:30 PM Mihai Budiu <[email protected]> wrote: > > > The typing rules for such types seem to be rather different from the rest > > of Calcite — if I understand correctly, these constructs are generally > > dynamically typed. So, I would be interested to see a design too prior to > > the implementation. > > > > Mihai > > ------------------------------ > > *From:* David Chaava via dev <[email protected]> > > *Sent:* Wednesday, November 4, 2026 6:19 AM > > *To:* [email protected] <[email protected]> > > *Cc:* David Chaava <[email protected]> > > *Subject:* Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > > > Hi Julian, Jerome, and Bertil, > > > > Thanks for the feedback and for sharing the implementation experience. > > > > Our immediate use case is preserving GEOGRAPHY through SQL validation and > > relational planning in Calcite-based engines such as Flink. A query may > > simply project an Iceberg geography column, so retaining its type is > useful > > even when no spatial functions are involved. > > > > Under the proposal, GEOGRAPHY would be a separate SQL type family. It > would > > not be a subtype of GEOMETRY in the argument-compatibility sense Julian > > described. An existing function declared to accept GEOMETRY would reject > a > > GEOGRAPHY argument during validation. This prevents a geography value > from > > reaching an implementation with planar semantics accidentally. > > > > For spatial functions, support would need to be considered individually, > > since some operations may share an implementation while others require > > geodetic algorithms. Runtime dispatch could be an option for a function > > explicitly declared to accept both types, provided its runtime > > representation preserves the distinction. The validator would still need > to > > check that the supplied argument types are supported. > > > > As Jerome points out, the distinction can also matter at planning time, > > when an adapter decides whether and how to push down an operation or an > > engine chooses an index. The first contribution would make the type > > distinction available to the planner. Pushdown rules and index selection > > are outside its scope. > > > > Our proposed contribution is independently useful for queries that carry > > GEOGRAPHY columns through a plan, with execution handled by the consuming > > engine. It does not depend on adding spatial functions to Calcite. Any > such > > functions would need their own discussion of semantics and implementation > > scope. > > > > On the SRID constraint, Bertil’s example explains our concern. A JTS > > geometry with SRID 4326 still uses planar edges and operations. > > Consequently, neither requiring an SRID nor fixing it to 4326 establishes > > geodetic semantics. A richer constrained-type model might express that > > distinction, and we understand the motivation to reduce duplicated > > type-system code. For this contribution, we propose using Calcite’s > > existing type machinery to express the distinction through a separate > type > > family. > > > > In the first contribution, Calcite’s GEOGRAPHY type would not encode CRS > or > > SRID in RelDataType, just as the existing GEOMETRY type does not encode > > SRID there. How Flink handles CRS/SRID, edge semantics, and encoding > > remains open in the FLIP discussion, where input from Sedona contributors > > would be welcome. > > > > The SQL type distinction is also compatible with sharing runtime > > components. Bertil’s suggestion to reuse JTS for representation, WKT/WKB > > handling, and suitable accessors illustrates how code could be shared. > > Jerome’s experience also helps identify the validation requirements. The > > first contribution would leave runtime representation and serialization > > unspecified. That keeps open the integration paths Jia outlined, > including > > RAW with Sedona-maintained adapters and a shared spatial type and SerDe. > > > > Regarding code size, we have a component breakdown rather than a reliable > > line-count estimate at this stage. The intended changes are concentrated > in > > the following areas. > > > > - SQL syntax and conformance support for the type name, parsing, and > > unparsing. > > - SqlTypeName, SqlTypeFamily, assignment and coercion rules, and the > > relevant type utilities and SQL/Rel conversion paths. > > - Regression tests and documentation for nullability, nested types, type > > propagation, mixed-type rejection, non-orderability, and SQL/Rel round > > trips. > > > > We expect to reuse the existing nullability and type-composition > machinery. > > The scope excludes spatial function implementations, a new Java value > > class, spatial library dependencies, and a general constrained-type > > framework. CALCITE-1466 (unsigned types) and CALCITE-4918 (VARIANT) are > > useful precedents for discussing the scope of type-system changes. A > > preliminary patch would help identify the changes needed in existing > > utilities and give us a concrete estimate of the size of the diff. > > > > Would this parser, validator, and relational type scope be acceptable for > > the first contribution? We would also appreciate feedback on the equality > > boundary already described in the proposal. It permits same-type > equality, > > with execution semantics and hashing supplied by the consuming engine or > > adapter. Are additional planner-level guarantees needed for > equality-based > > operations such as DISTINCT and GROUP BY, particularly when those > > operations may execute locally? > > > > Best, > > David > > > > On Wed, Nov 4, 2026 at 11:14 AM Jerome Haltom <[email protected] > > > > wrote: > > > > > In my case, I am primarily trying to decide whether or not I can push > an > > > operation down or how to push the operation down. > > > > > > For an actual engine, knowing it at plan time will matter for index > > > selection, etc. > > > > > > > > > > > > Get Outlook for Android<https://aka.ms/AAb9ysg> > > > ________________________________ > > > From: Julian Hyde <[email protected]> > > > Sent: Tuesday, 03 November 2026 13:05:16 > > > To: [email protected] <[email protected]> > > > Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > > > > > Is there actually a requirement to overload? The function could check > at > > > runtime whether it has a GEOMETRY or GEOGRAPHY value. > > > > > > > On Nov 3, 2026, at 10:54 AM, Jerome Haltom <[email protected] > > > > > wrote: > > > > > > > > It has been a couple months since I had thought through this, but I > > > think the downside of my approach was there was no way to overload > > > functions in a way that made sense. I don't think you could overload by > > > constraint. > > > > > > > > So, so you couldn't validate. Which most engines do. > > > > > > > > ________________________________ > > > > From: Julian Hyde <[email protected]> > > > > Sent: Tuesday, November 03, 2026 12:11 > > > > To: [email protected] <[email protected]> > > > > Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > > > > > > > There are a lot of geospatial functions. Would the functions that > > > currently take GEOMETRY also be applicable to GEOGRAPHY? Do we need > > > additional functions for the GEOGRAPHY type? > > > > > > > > Is GEOGRAPHY a subtype if GEOMETRY (in the sense that a GEOGRAPHY > value > > > can be used as an argument to a function with a GEOMETRY parameter? > > > > > > > > I worry about this change creating a lot of new code (possibly > > > copy-pasted from existing code, possibly generated by agents, but > > > nevertheless requiring maintenance). Give me a sense of the amount of > new > > > code that would be required, and which components it would reside in. > > > > > > > > I wonder whether GEOGRAPHY could be a GEOMETRY type with a constraint > > > “srid IS NOT NULL”. We cannot currently attach constraints to types > > (except > > > the constraints that are represented as type parameters, e.g. > VARCHAR(10) > > > is “a string whose length is not greater than 10”). Is a constrained > > > GEOMETRY type the right way to think of this? If so, attaching > > constraints > > > to types might be a way to reduce the amount of code that has to be > > written. > > > > > > > > (I had a similar thought about constraints and types when we had the > > > discussion about the VECTOR type. I still believe that the VECTOR type > is > > > basically an ARRAY type where the length is a constant. My work on > Morel, > > > and its elegant type system based on Standard ML’s modules and > functors, > > > makes me think about how we remove/reduce code by improving our type > > > system.) > > > > > > > > Julian > > > > > > > > > > > >> On Nov 3, 2026, at 8:24 AM, Jerome Haltom <[email protected] > > > > > wrote: > > > >> > > > >> I approve of this, as it would let me deprecate my existing > > > geography-like support. > > > >> > > > >> > > > > > > https://github.com/ikvmnet/calcite-dotnet/tree/main/src/Apache.Calcite.Geography > > > >> > > > >> This will be weird for you guys, since it's .NET code. But it's an > > > attempt at a full suite of Geography functions across the existing > > geometry > > > types. It is only WGS84 however. It uses S2 and > > > net.sf.geographiclib:GeographicLib-Java > > > >> > > > >> > > > >> > > > >> ________________________________ > > > >> From: David Chaava via dev <[email protected]> > > > >> Sent: Tuesday, November 03, 2026 08:51 > > > >> To: [email protected] <[email protected]> > > > >> Cc: [email protected] <[email protected]>; [email protected] < > > > [email protected]>; David Chaava <[email protected]> > > > >> Subject: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > > >> > > > >> Hi Calcite community, > > > >> > > > >> We would like to discuss adding GEOGRAPHY as a first-class SQL and > > > >> relational type in Calcite, distinct from the existing GEOMETRY > type. > > > >> > > > >> This follows the discussion about native GEOGRAPHY support in Flink > > [1], > > > >> motivated in part by interoperability with Iceberg geography > columns. > > > >> Participants raised concerns about maintaining a Flink-only planner > > > type, > > > >> handling validation and type-system edge cases, and eventually > > > migrating to > > > >> an upstream Calcite type. We would like to address these concerns > > > through a > > > >> Calcite contribution. > > > >> > > > >> Our proposal is available here [2]. > > > >> > > > >> The proposed first change covers parsing, validation, relational > type > > > >> propagation, and unparsing, including nullability, type coercion > > rules, > > > >> nested types, and SQL/Rel round trips. A separate type family would > > > >> preserve the distinction between geographic and planar values and > > > prevent > > > >> existing GEOMETRY functions from accepting GEOGRAPHY operands > > > >> unintentionally. > > > >> > > > >> The initial type would be unparameterized and independent of a > > > particular > > > >> runtime representation. The proposal leaves CRS/SRID metadata, > > encoding, > > > >> and runtime execution to consuming engines and adapters. It does not > > > >> introduce spatial functions, constructors, > > > serialization/deserialization, > > > >> or storage-format mappings. > > > >> > > > >> We also want to build on the Sedona discussion. Jia clarified that > > > >> SedonaFlink already supports Geography through RAW with a dedicated > > > >> serializer, and called for joint consideration of CRS/SRID, > > > spherical-edge > > > >> semantics, validation, equality, and shared serialization. The > limited > > > >> Calcite scope proposed here is intended as a first step; it does not > > > settle > > > >> those broader interoperability choices or assume that Sedona would > > > adopt a > > > >> particular runtime representation. > > > >> > > > >> We would appreciate feedback on two main questions: > > > >> > > > >> 1. Should Calcite expose an unparameterized, representation-neutral > > > >> GEOGRAPHY type distinct from GEOMETRY? > > > >> 2. Is the proposed parser/validator/type-system scope sufficient > for > > a > > > >> first contribution without constructors or a Calcite runtime value > > > class? > > > >> > > > >> In considering these questions, we would particularly welcome > feedback > > > on > > > >> the proposed semantic boundary: same-type equality is permitted with > > > >> adapter-defined execution semantics, ordering is rejected, and > > CRS/SRID > > > is > > > >> not encoded in the Calcite type. Are additional guarantees needed at > > the > > > >> planner level? > > > >> > > > >> Subject to Calcite review and acceptance, the intended integration > > path > > > is > > > >> to merge the necessary support upstream, then bring those changes > into > > > >> Flink, with any temporary copies removed when Flink upgrades its > > Calcite > > > >> dependency. We would also welcome suggestions for regression cases, > > > >> particularly around null handling, nested types, and type-to-SQL > > > conversion. > > > >> > > > >> The Flink dev list and Jia are copied to help coordinate the > > discussion. > > > >> Thank you for your feedback. > > > >> > > > >> Best regards, > > > >> David > > > >> > > > >> [1] Flink GEOGRAPHY discussion > > > >> <https://lists.apache.org/thread/flbj8dfdfgs26klrxt7xch3r9785ky67> > > > >> [2] Planner-level GEOGRAPHY type in Apache Calcite proposal > > > >> < > > > > > > https://docs.google.com/document/d/1Btqd0oZT8ctqIaGI_AROyAFJQe68RsVyeQbtmK4wHQw/edit > > > > > > > > > > > > > > > > >
