[
https://issues.apache.org/jira/browse/SPARK-50710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-50710:
------------------------------------
Assignee: Venkata Sai Akhil Gudesa
> Add support for optional client reconnection to sessions after release
> ----------------------------------------------------------------------
>
> Key: SPARK-50710
> URL: https://issues.apache.org/jira/browse/SPARK-50710
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: 4.0.0
> Reporter: Venkata Sai Akhil Gudesa
> Assignee: Venkata Sai Akhil Gudesa
> Priority: Major
> Labels: pull-request-available
>
> Currently, the connect server will, by default, tombstone all sessions that
> have either been released explicitly (through a `ReleaseSession` request) or
> cleaned up due to inactivity/idleness in periodic checks.
> Tombstoning prevents clients from reconnecting with the same `userId` and
> `sessionId`. This mechanism ensures that clients do not accidentally end up
> with a 'fresh' server-side session, which may be disastrous/fatal as all
> previously held state is lost (e.g., Temporary views, temporary UDFs,
> modified configs, current catalog, etc.).
>
> Consider a client that runs simple non-state dependant queries (e.g `select
> count(*) from ...`), perhaps sparsely during the lifetime of the application.
> Such a client may prefer to opt out of tombstoning for the following reasons:
> * Queries do not depend on any custom server-side state.
> * Modifying `userId`/`sessionId` on each reconnect may be inconvenient for
> tracking/observability purposes.
> * On resource-constrained servers, clients may want to minimize their memory
> footprint by explicitly releasing their state, especially when they believe
> their requests are sparsely spread out.
>
> Currently, the only way to allow clients to reconnect is to set `
> spark.connect.session.manager.closedSessionsTombstonesSize` to `0`. However,
> this is not ideal as it would allow *all* clients to reconnect, which as
> previously pointed out, may be dangerous.
>
> As an improvement, allowing specific clients to explicitly signal/request the
> reconnection possibility addresses the needs mentioned earlier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
