Re: [DISCUSS] SPIP: Add geospatial types to Spark

2025-03-28 Thread Menelaos Karavelas
rez Pascua > (mailto:angel.alvarez.pas...@gmail.com>>) > escribió: >> +1 (non-binding) >> >> El vie, 28 mar 2025, 18:48, Menelaos Karavelas > <mailto:menelaos.karave...@gmail.com>> escribió: >>> Dear Spark community, >>> >>> I w

[VOTE] SPIP: Add geospatial types to Spark

2025-05-05 Thread Menelaos Karavelas
proposal as an official SPIP [ ] +0 [ ] -1: I don’t think this is a good idea because Menelaos Karavelas

Re: [VOTE] SPIP: Add geospatial types to Spark

2025-05-06 Thread Menelaos Karavelas
a...@gmail.com>>: >>>>>>>>>>> +1 (non-binding) >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Milan >>>>>>>>>

Re: [DISCUSS] SPIP: Add geospatial types to Spark

2025-03-29 Thread Menelaos Karavelas
Menelaus may > explain more about the SPIP goal. > > I do hope there can be more collaboration across communities (like in > Iceberg/Parquet collaboration) in getting Sedona community’s experience in > making sure these type definitions are optimal , and compatible for Sedona. >

Re: [DISCUSS] SPIP: Add geospatial types to Spark

2025-03-29 Thread Menelaos Karavelas
ts SQL: DBSCAN, hotspot analysis, outlier detection >> • Language support: >> • Scala, Java, SQL, Python, R >> • Query acceleration via R-Tree: >> • Distributed and broadcast spatial joins >> • KNN joins >>

[DISCUSS] SPIP: Add geospatial types to Spark

2025-03-28 Thread Menelaos Karavelas
types and perform some minimal meaningful operations on them. SPIP: https://issues.apache.org/jira/browse/SPARK-51658 Looking forward to your comments and feedback. Best regards, Menelaos Karavelas

[VOTE][RESULT] SPIP: Add geospatial types to Spark

2025-05-08 Thread Menelaos Karavelas
The vote passes with 19 +1s (12 binding +1s) and one +0. Thank you to all that have participated in the discussion and the vote and for all the feedback! (* = binding) +1: - DB Tsai * - Szehon Ho - Milan Stefanovic - Bjørn Jørgensen - Reynold Xin * - Wenchen Fan * - Denny Lee - Yuming Wang * - Xi

Re: [DISCUSS] Proposal to Add Theta and Tuple Sketches to Spark SQL

2025-06-03 Thread Menelaos Karavelas
Hello Chris. HLL sketches from the same project (Apache DataSketches) have already been integrated in Spark. How does your proposal fit given what I just mentioned? - Menelaos > On Jun 3, 2025, at 2:52 PM, Boumalhab, Chris > wrote: > > Hi all, > > I’d like to start a discussion about addi

Re: [DISCUSS] Proposal to Add Theta and Tuple Sketches to Spark SQL

2025-06-03 Thread Menelaos Karavelas
os, > > Thanks for pointing that out. HLL sketches do not support set operations such > as intersection or difference. Tuple sketches would also allow value > aggregation for the same key. For those reasons, I don’t believe HLL is > enough. > > Chris > > Fro

Re: [DISCUSS] Proposal to Add Theta and Tuple Sketches to Spark SQL

2025-06-03 Thread Menelaos Karavelas
Following what Ryan did for HLL sketches, I would also add an aggregate expression for unions as the aggregate version of the binary union expression. The expressions that Ryan added are: hll_sketch_agg hll_union hll_union_agg hll_sketch_estimate Following the same naming convention I would prob