zhangfengcdt commented on code in PR #362:
URL: https://github.com/apache/sedona-db/pull/362#discussion_r2585886974
##########
rust/sedona-spatial-join/src/build_index.rs:
##########
@@ -33,7 +33,72 @@ use crate::{
spatial_predicate::SpatialPredicate,
};
-pub(crate) async fn build_index(
+/// Synchronous version of build_index that doesn't spawn tasks
+/// Used in execution contexts without async runtime support (e.g.,
Spark/Comet JNI)
+pub async fn build_index_sync(
Review Comment:
I see your tests directly places JoinSet::spawn directly inside poll_next.
The hang I observed might not be directly caused by JoinSet::spawn itself. The
difference might be:
- Your test: Spawns directly inside poll_next using the provided Context
- Spatial join: Uses OnceAsync for lazy initialization, which has its own
coordination logic
The interaction between block_on(async { poll!(...) }), OnceAsync, and
JoinSet might behave differently than direct spawning in poll_next.
But in SpatialJoinExec, build_index is called through OnceAsync, which has
its own coordination logic. The execution path is:
1. JNI calls get_runtime().block_on(async { poll!(stream.next()) })
2. Stream's poll_next triggers OnceAsync
3. OnceAsync calls build_index → collect_all → JoinSet::spawn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
