[
https://issues.apache.org/jira/browse/HDDS-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18128955#comment-18128955
]
Andy Huang edited comment on HDDS-17140 at 10/20/26 1:47 AM:
-------------------------------------------------------------
Thanks [~sodonnell] and [~erose]. After looking through the client code, I have
two questions about the refresh behavior:
# Would refreshing on demand after a 10-minute cache expiry be acceptable,
rather than polling in the background? RpcClient.getServerDefaults() already
uses this pattern. It avoids polling idle clients, although the first operation
requiring version information after expiry would incur an additional RPC. It
also avoids an S3 authentication issue: when s3AuthCheck is enabled, the
client-side translator rejects requests without thread-local S3Auth, including
ServiceList, so a background refresh cannot simply call the existing
getServiceInfo().
# If a refresh fails, would it be reasonable to retain the last successfully
retrieved OM version and retry with throttling? This would allow existing
operations to continue during a temporary failure without making every
subsequent request attempt another refresh.
For scope, I propose updating the OM version used by client feature checks
without recreating the OzoneClient or changing the existing server-defaults
cache. Existing output streams would retain the version captured when they were
created; subsequent operations and newly created streams would use the
refreshed version.
Does this approach match what you had in mind?
was (Author: JIRAUSER314249):
Thanks Stephen and Ethan. After looking through the client code, I have two
questions about the refresh behavior:
# Would refreshing on demand after a 10-minute cache expiry be acceptable,
rather than polling in the background? RpcClient.getServerDefaults() already
uses this pattern. It avoids polling idle clients, although the first operation
requiring version information after expiry would incur an additional RPC. It
also avoids an S3 authentication issue: when s3AuthCheck is enabled, the
client-side translator rejects requests without thread-local S3Auth, including
ServiceList, so a background refresh cannot simply call the existing
getServiceInfo().
# If a refresh fails, would it be reasonable to retain the last successfully
retrieved OM version and retry with throttling? This would allow existing
operations to continue during a temporary failure without making every
subsequent request attempt another refresh.
For scope, I propose updating the OM version used by client feature checks
without recreating the OzoneClient or changing the existing server-defaults
cache. Existing output streams would retain the version captured when they were
created; subsequent operations and newly created streams would use the
refreshed version.
Does this approach match what you had in mind?
> S3 Gateway does not pick up the new OM version after ZDU finalization
> ----------------------------------------------------------------------
>
> Key: HDDS-17140
> URL: https://issues.apache.org/jira/browse/HDDS-17140
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Chung-En Lee
> Assignee: Andy Huang
> Priority: Major
>
> The ZDU design doc ("Order of Operations During the Upgrade", step 5) says to
> rolling-restart S3 Gateway and other client processes on the new software
> before the cluster is finalized. At that point OM is still pre-finalized, so
> it reports the old version to clients.
> S3 Gateway learns the OM version only once, when it handles its first request
> after starting, and keeps it for the life of the process. In a ZDU rolling
> restart S3 Gateway keeps serving traffic, so it captures the old version
> before finalization. After the admin finalizes the cluster, OM reports the
> new version, but S3 Gateway keeps the old one. Any new feature that depends
> on the finalized OM version stays disabled in S3 Gateway until it is
> restarted again.
> Proposed fix (options):
> 1. Change the documented upgrade order so S3 Gateway is restarted after
> finalization.
> 2. Keep the current order, and document that S3 Gateway must be restarted
> again after finalization.
> 3. Make S3 Gateway aware OM version changes (e.g. after finalization) and
> re-initialize its OzoneClient automatically, so it picks up the latest OM
> version without a manual restart.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
