Hi Mihai and Julian, Thanks for clarifying the distinction between the static SQL type and the shape of the value it contains.
We explored that direction in our local Calcite experiment. For basic GEOGRAPHY support, we already reuse BasicSqlType, SqlTypeFactoryImpl.createSqlType(), and SqlBasicTypeNameSpec alongside GEOMETRY. The additions cover type and family registration, parser/conformance support, and compatibility rules. In the tested ST_Area case, the existing FamilyOperandTypeChecker rejects GEOGRAPHY without changes to the checker. We then built a test-only extension using one spatial type-description class with shape and coordinate-layout properties. Both versions retain separate GEOMETRY and GEOGRAPHY SQL names and families. The extension explores independent shape/layout guarantees within a shared implementation. Whether constraints could replace the separate family distinction remains unexplored. The prototype tracks SQL type, shape, and coordinate layout separately. A test schema supplies guarantees about all non-null values, such as “every value is a point with two coordinate components.” XY means two components per vertex; XYZ includes a third component, Z. The presence of Z does not determine whether the SQL type is GEOMETRY or GEOGRAPHY. For example, suppose columns `a` and `b` both have SQL type GEOGRAPHY. When deriving the type of CASE WHEN flag THEN a ELSE b END: - If `a` is guaranteed to contain points with XY components and `b` LineStrings with XY components, the result remains GEOGRAPHY with an XY guarantee, while its shape becomes unknown. - If both columns are guaranteed to contain points, but `a` has XY components and `b` XYZ components, the result retains the point guarantee, while its coordinate layout becomes unknown. These rules relax the static guarantees. They do not introduce a conversion that removes coordinate components. The tests cover type inference and planning; runtime execution remains outside the experiment. The targeted tests exercise these rules through CASE/COALESCE, UNION ALL, nested ARRAY/ROW types, nullable copies, and project removal. The shape/layout properties come from test-schema metadata, with SQL syntax and runtime enforcement still to be designed. We could reuse the existing container inference paths and plan handling. Independent properties additionally required common-type inference, preservation through copying and type identity, and preservation of full inferred targets through internal casts. In particular, SqlTypeUtil.convertTypeToSpec retains the SQL name but does not encode shape/layout. Array constructors needed a separate hook because they insert casts outside the usual coercion path. A validation-only checker exercised the point guarantee. These results show how a shared representation can carry the additional properties in the tested paths. General cast/assignment compatibility and preservation of those properties in saved plans still need design. We have not measured a net reduction in code or maintenance cost. Julian, does this reflect the direction you had in mind? Our proposed first step is the basic GEOGRAPHY type using the existing shared infrastructure. We also understand your point about matching the PostGIS function set. Would you be comfortable with an initial contribution covering parsing, validation, relational type propagation, and unparsing, with function coverage and independent shape/layout constraints scoped separately? Best, David On Fri, Nov 6, 2026 at 6:58 PM Julian Hyde <[email protected]> wrote: > Yes, making GEOMETRY and GEOGRAPHY static types, and shape type (e.g. > point vs line) and dimensionality (2D vs 3D) is the decision that PostGIS > made. We should stick with that decision. Concretely, that means that we > should add a lot of functions with GEOGRAPHY parameters to match the > PostGIS function set. > > But I want to point out that that split — into two types rather than a > couple dozen, for every shape type and dimensionality — was a rather > arbitrary decision. There are optimizations that we could do if we knew > that we were always dealing with points or lines. And repeating the > plumbing in the validator and planner for a new type might involve a lot of > copy-paste. > > I believe that adding constraints to our internal representation of the > GEOMETRY type might allow us to represent geographies and also all of the > shape types. > > > On Nov 6, 2026, at 9:45 AM, Mihai Budiu <[email protected]> wrote: > > > > > > Exactly, ST_LINE and ST_POINT are the dynamic type of an object with a > GEOMETRY/GEOGRAPHY static type. > > The VARIANT is similar. > > > > The VARIANT implementation is a proof that this can be done within the > framework. > > > > Mihai > > > > ________________________________ > > From: David Chaava via dev <[email protected]> > > Sent: Friday, November 6, 2026 5:02 AM > > To: [email protected] <[email protected]> > > Cc: David Chaava <[email protected]> > > Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > > > > Hi Mihai, > > > > Thanks for taking a look. Let me make the proposed typing rules more > > concrete. > > > > Suppose `places` has a nullable column named `g` of SQL type `GEOGRAPHY`, > > and the spatial operator table is enabled. Under the proposal, the > > following queries would behave as shown. > > > > SELECT g FROM places > > -- The result column has type GEOGRAPHY. > > > > SELECT COALESCE(g, NULL) FROM places > > -- The result column has type GEOGRAPHY and is nullable. > > > > SELECT ST_Area(g) FROM places > > -- Validation fails because the existing ST_Area expects GEOMETRY. > > > > At validation time, Calcite knows from the schema that `g` has SQL type > > `GEOGRAPHY`. It uses that type to determine the result type of the first > > two queries and to check whether `g` matches the declared `ST_Area` > > signature in the third. These checks happen before Calcite reads any > rows. > > A point, a line, and a polygon are different shapes that a `GEOGRAPHY` > > value can represent, and all three have the same SQL type in this design. > > By comparison, a `VARIANT` value carries the SQL type of its contents at > > runtime. We cited CALCITE-4918 as a precedent for separating parser and > > validator work from runtime work. > > > > If you had a particular expression or operation in mind when drawing the > > analogy with `VARIANT`, we'd be happy to walk through how the proposed > > typing rules would apply to it. > > > > I hope this helps clarify the design. If you have any other questions, > we'd > > be happy to walk through more examples. > > > > Best, > > David > > > > On Fri, Nov 6, 2026 at 12:31 AM Mihai Budiu <[email protected]> wrote: > > > >> I looked, but I am not much more enlightened. > >> > >> Maybe GEOMETRY/GEOGRAPHY will work a little like specialized forms > >> VARIANT, which is really dynamically-typed. > >> > >> Mihai > >> > >> ------------------------------ > >> *From:* David Chaava via dev <[email protected]> > >> *Sent:* Thursday, November 5, 2026 3:45 AM > >> *To:* [email protected] <[email protected]> > >> *Cc:* David Chaava <[email protected]> > >> *Subject:* Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > >> > >> Hi Mihai, > >> > >> Thanks for raising this. We have a design proposal available here: > >> > >> > https://docs.google.com/document/d/1Btqd0oZT8ctqIaGI_AROyAFJQe68RsVyeQbtmK4wHQw/edit > >> > >> We would appreciate your feedback on it before implementation. > >> > >> The proposal introduces GEOGRAPHY as a statically known SQL and > relational > >> type. A column would be declared as GEOGRAPHY by its schema or adapter, > and > >> Calcite would preserve that type through validation and planning. Point, > >> line, and polygon values would all have the same SQL type in this > initial > >> design. > >> > >> The runtime dispatch mentioned earlier was a possibility for spatial > >> functions explicitly supporting both GEOMETRY and GEOGRAPHY. It would > still > >> require validation of the declared argument types. The first > contribution > >> focuses on the type itself and its propagation through a plan. > >> > >> The proposal includes assignment and coercion rules, nullability, nested > >> types, and the intended behavior of comparisons and set operations. > >> Equality and grouping are among the areas where we are seeking feedback > on > >> the guarantees Calcite needs. > >> > >> Could you help us understand which aspects you have in mind when > referring > >> to dynamic typing? We would appreciate your guidance on where the > proposed > >> typing rules need further clarification. > >> > >> Best, > >> David > >> > >> On Wed, Nov 4, 2026 at 8:30 PM Mihai Budiu <[email protected]> wrote: > >> > >>> The typing rules for such types seem to be rather different from the > rest > >>> of Calcite — if I understand correctly, these constructs are generally > >>> dynamically typed. So, I would be interested to see a design too prior > to > >>> the implementation. > >>> > >>> Mihai > >>> ------------------------------ > >>> *From:* David Chaava via dev <[email protected]> > >>> *Sent:* Wednesday, November 4, 2026 6:19 AM > >>> *To:* [email protected] <[email protected]> > >>> *Cc:* David Chaava <[email protected]> > >>> *Subject:* Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > >>> > >>> Hi Julian, Jerome, and Bertil, > >>> > >>> Thanks for the feedback and for sharing the implementation experience. > >>> > >>> Our immediate use case is preserving GEOGRAPHY through SQL validation > and > >>> relational planning in Calcite-based engines such as Flink. A query may > >>> simply project an Iceberg geography column, so retaining its type is > >> useful > >>> even when no spatial functions are involved. > >>> > >>> Under the proposal, GEOGRAPHY would be a separate SQL type family. It > >> would > >>> not be a subtype of GEOMETRY in the argument-compatibility sense Julian > >>> described. An existing function declared to accept GEOMETRY would > reject > >> a > >>> GEOGRAPHY argument during validation. This prevents a geography value > >> from > >>> reaching an implementation with planar semantics accidentally. > >>> > >>> For spatial functions, support would need to be considered > individually, > >>> since some operations may share an implementation while others require > >>> geodetic algorithms. Runtime dispatch could be an option for a function > >>> explicitly declared to accept both types, provided its runtime > >>> representation preserves the distinction. The validator would still > need > >> to > >>> check that the supplied argument types are supported. > >>> > >>> As Jerome points out, the distinction can also matter at planning time, > >>> when an adapter decides whether and how to push down an operation or an > >>> engine chooses an index. The first contribution would make the type > >>> distinction available to the planner. Pushdown rules and index > selection > >>> are outside its scope. > >>> > >>> Our proposed contribution is independently useful for queries that > carry > >>> GEOGRAPHY columns through a plan, with execution handled by the > consuming > >>> engine. It does not depend on adding spatial functions to Calcite. Any > >> such > >>> functions would need their own discussion of semantics and > implementation > >>> scope. > >>> > >>> On the SRID constraint, Bertil’s example explains our concern. A JTS > >>> geometry with SRID 4326 still uses planar edges and operations. > >>> Consequently, neither requiring an SRID nor fixing it to 4326 > establishes > >>> geodetic semantics. A richer constrained-type model might express that > >>> distinction, and we understand the motivation to reduce duplicated > >>> type-system code. For this contribution, we propose using Calcite’s > >>> existing type machinery to express the distinction through a separate > >> type > >>> family. > >>> > >>> In the first contribution, Calcite’s GEOGRAPHY type would not encode > CRS > >> or > >>> SRID in RelDataType, just as the existing GEOMETRY type does not encode > >>> SRID there. How Flink handles CRS/SRID, edge semantics, and encoding > >>> remains open in the FLIP discussion, where input from Sedona > contributors > >>> would be welcome. > >>> > >>> The SQL type distinction is also compatible with sharing runtime > >>> components. Bertil’s suggestion to reuse JTS for representation, > WKT/WKB > >>> handling, and suitable accessors illustrates how code could be shared. > >>> Jerome’s experience also helps identify the validation requirements. > The > >>> first contribution would leave runtime representation and serialization > >>> unspecified. That keeps open the integration paths Jia outlined, > >> including > >>> RAW with Sedona-maintained adapters and a shared spatial type and > SerDe. > >>> > >>> Regarding code size, we have a component breakdown rather than a > reliable > >>> line-count estimate at this stage. The intended changes are > concentrated > >> in > >>> the following areas. > >>> > >>> - SQL syntax and conformance support for the type name, parsing, and > >>> unparsing. > >>> - SqlTypeName, SqlTypeFamily, assignment and coercion rules, and the > >>> relevant type utilities and SQL/Rel conversion paths. > >>> - Regression tests and documentation for nullability, nested types, > type > >>> propagation, mixed-type rejection, non-orderability, and SQL/Rel round > >>> trips. > >>> > >>> We expect to reuse the existing nullability and type-composition > >> machinery. > >>> The scope excludes spatial function implementations, a new Java value > >>> class, spatial library dependencies, and a general constrained-type > >>> framework. CALCITE-1466 (unsigned types) and CALCITE-4918 (VARIANT) are > >>> useful precedents for discussing the scope of type-system changes. A > >>> preliminary patch would help identify the changes needed in existing > >>> utilities and give us a concrete estimate of the size of the diff. > >>> > >>> Would this parser, validator, and relational type scope be acceptable > for > >>> the first contribution? We would also appreciate feedback on the > equality > >>> boundary already described in the proposal. It permits same-type > >> equality, > >>> with execution semantics and hashing supplied by the consuming engine > or > >>> adapter. Are additional planner-level guarantees needed for > >> equality-based > >>> operations such as DISTINCT and GROUP BY, particularly when those > >>> operations may execute locally? > >>> > >>> Best, > >>> David > >>> > >>> On Wed, Nov 4, 2026 at 11:14 AM Jerome Haltom > <[email protected] > >>> > >>> wrote: > >>> > >>>> In my case, I am primarily trying to decide whether or not I can push > >> an > >>>> operation down or how to push the operation down. > >>>> > >>>> For an actual engine, knowing it at plan time will matter for index > >>>> selection, etc. > >>>> > >>>> > >>>> > >>>> Get Outlook for Android<https://aka.ms/AAb9ysg> > >>>> ________________________________ > >>>> From: Julian Hyde <[email protected]> > >>>> Sent: Tuesday, 03 November 2026 13:05:16 > >>>> To: [email protected] <[email protected]> > >>>> Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > >>>> > >>>> Is there actually a requirement to overload? The function could check > >> at > >>>> runtime whether it has a GEOMETRY or GEOGRAPHY value. > >>>> > >>>>> On Nov 3, 2026, at 10:54 AM, Jerome Haltom <[email protected] > >>> > >>>> wrote: > >>>>> > >>>>> It has been a couple months since I had thought through this, but I > >>>> think the downside of my approach was there was no way to overload > >>>> functions in a way that made sense. I don't think you could overload > by > >>>> constraint. > >>>>> > >>>>> So, so you couldn't validate. Which most engines do. > >>>>> > >>>>> ________________________________ > >>>>> From: Julian Hyde <[email protected]> > >>>>> Sent: Tuesday, November 03, 2026 12:11 > >>>>> To: [email protected] <[email protected]> > >>>>> Subject: Re: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > >>>>> > >>>>> There are a lot of geospatial functions. Would the functions that > >>>> currently take GEOMETRY also be applicable to GEOGRAPHY? Do we need > >>>> additional functions for the GEOGRAPHY type? > >>>>> > >>>>> Is GEOGRAPHY a subtype if GEOMETRY (in the sense that a GEOGRAPHY > >> value > >>>> can be used as an argument to a function with a GEOMETRY parameter? > >>>>> > >>>>> I worry about this change creating a lot of new code (possibly > >>>> copy-pasted from existing code, possibly generated by agents, but > >>>> nevertheless requiring maintenance). Give me a sense of the amount of > >> new > >>>> code that would be required, and which components it would reside in. > >>>>> > >>>>> I wonder whether GEOGRAPHY could be a GEOMETRY type with a constraint > >>>> “srid IS NOT NULL”. We cannot currently attach constraints to types > >>> (except > >>>> the constraints that are represented as type parameters, e.g. > >> VARCHAR(10) > >>>> is “a string whose length is not greater than 10”). Is a constrained > >>>> GEOMETRY type the right way to think of this? If so, attaching > >>> constraints > >>>> to types might be a way to reduce the amount of code that has to be > >>> written. > >>>>> > >>>>> (I had a similar thought about constraints and types when we had the > >>>> discussion about the VECTOR type. I still believe that the VECTOR type > >> is > >>>> basically an ARRAY type where the length is a constant. My work on > >> Morel, > >>>> and its elegant type system based on Standard ML’s modules and > >> functors, > >>>> makes me think about how we remove/reduce code by improving our type > >>>> system.) > >>>>> > >>>>> Julian > >>>>> > >>>>> > >>>>>> On Nov 3, 2026, at 8:24 AM, Jerome Haltom <[email protected] > >>> > >>>> wrote: > >>>>>> > >>>>>> I approve of this, as it would let me deprecate my existing > >>>> geography-like support. > >>>>>> > >>>>>> > >>>> > >>> > >> > https://github.com/ikvmnet/calcite-dotnet/tree/main/src/Apache.Calcite.Geography > >>>>>> > >>>>>> This will be weird for you guys, since it's .NET code. But it's an > >>>> attempt at a full suite of Geography functions across the existing > >>> geometry > >>>> types. It is only WGS84 however. It uses S2 and > >>>> net.sf.geographiclib:GeographicLib-Java > >>>>>> > >>>>>> > >>>>>> > >>>>>> ________________________________ > >>>>>> From: David Chaava via dev <[email protected]> > >>>>>> Sent: Tuesday, November 03, 2026 08:51 > >>>>>> To: [email protected] <[email protected]> > >>>>>> Cc: [email protected] <[email protected]>; [email protected] < > >>>> [email protected]>; David Chaava <[email protected]> > >>>>>> Subject: [DISCUSS] Planner-level GEOGRAPHY type in Apache Calcite > >>>>>> > >>>>>> Hi Calcite community, > >>>>>> > >>>>>> We would like to discuss adding GEOGRAPHY as a first-class SQL and > >>>>>> relational type in Calcite, distinct from the existing GEOMETRY > >> type. > >>>>>> > >>>>>> This follows the discussion about native GEOGRAPHY support in Flink > >>> [1], > >>>>>> motivated in part by interoperability with Iceberg geography > >> columns. > >>>>>> Participants raised concerns about maintaining a Flink-only planner > >>>> type, > >>>>>> handling validation and type-system edge cases, and eventually > >>>> migrating to > >>>>>> an upstream Calcite type. We would like to address these concerns > >>>> through a > >>>>>> Calcite contribution. > >>>>>> > >>>>>> Our proposal is available here [2]. > >>>>>> > >>>>>> The proposed first change covers parsing, validation, relational > >> type > >>>>>> propagation, and unparsing, including nullability, type coercion > >>> rules, > >>>>>> nested types, and SQL/Rel round trips. A separate type family would > >>>>>> preserve the distinction between geographic and planar values and > >>>> prevent > >>>>>> existing GEOMETRY functions from accepting GEOGRAPHY operands > >>>>>> unintentionally. > >>>>>> > >>>>>> The initial type would be unparameterized and independent of a > >>>> particular > >>>>>> runtime representation. The proposal leaves CRS/SRID metadata, > >>> encoding, > >>>>>> and runtime execution to consuming engines and adapters. It does not > >>>>>> introduce spatial functions, constructors, > >>>> serialization/deserialization, > >>>>>> or storage-format mappings. > >>>>>> > >>>>>> We also want to build on the Sedona discussion. Jia clarified that > >>>>>> SedonaFlink already supports Geography through RAW with a dedicated > >>>>>> serializer, and called for joint consideration of CRS/SRID, > >>>> spherical-edge > >>>>>> semantics, validation, equality, and shared serialization. The > >> limited > >>>>>> Calcite scope proposed here is intended as a first step; it does not > >>>> settle > >>>>>> those broader interoperability choices or assume that Sedona would > >>>> adopt a > >>>>>> particular runtime representation. > >>>>>> > >>>>>> We would appreciate feedback on two main questions: > >>>>>> > >>>>>> 1. Should Calcite expose an unparameterized, representation-neutral > >>>>>> GEOGRAPHY type distinct from GEOMETRY? > >>>>>> 2. Is the proposed parser/validator/type-system scope sufficient > >> for > >>> a > >>>>>> first contribution without constructors or a Calcite runtime value > >>>> class? > >>>>>> > >>>>>> In considering these questions, we would particularly welcome > >> feedback > >>>> on > >>>>>> the proposed semantic boundary: same-type equality is permitted with > >>>>>> adapter-defined execution semantics, ordering is rejected, and > >>> CRS/SRID > >>>> is > >>>>>> not encoded in the Calcite type. Are additional guarantees needed at > >>> the > >>>>>> planner level? > >>>>>> > >>>>>> Subject to Calcite review and acceptance, the intended integration > >>> path > >>>> is > >>>>>> to merge the necessary support upstream, then bring those changes > >> into > >>>>>> Flink, with any temporary copies removed when Flink upgrades its > >>> Calcite > >>>>>> dependency. We would also welcome suggestions for regression cases, > >>>>>> particularly around null handling, nested types, and type-to-SQL > >>>> conversion. > >>>>>> > >>>>>> The Flink dev list and Jia are copied to help coordinate the > >>> discussion. > >>>>>> Thank you for your feedback. > >>>>>> > >>>>>> Best regards, > >>>>>> David > >>>>>> > >>>>>> [1] Flink GEOGRAPHY discussion > >>>>>> <https://lists.apache.org/thread/flbj8dfdfgs26klrxt7xch3r9785ky67> > >>>>>> [2] Planner-level GEOGRAPHY type in Apache Calcite proposal > >>>>>> < > >>>> > >>> > >> > https://docs.google.com/document/d/1Btqd0oZT8ctqIaGI_AROyAFJQe68RsVyeQbtmK4wHQw/edit > >>>>> > >>>>> > >>>> > >>>> > >>> > >> > >
