[
https://issues.apache.org/jira/browse/HDDS-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18128911#comment-18128911
]
Ethan Rose commented on HDDS-17140:
-----------------------------------
I agree it would be best to handle this generically within the client, since it
also applies to long living applications. Generic non-s3g clients can be newer
than the OM server regardless of OM's upgrade status which raises two points:
* Newer clients may switch to newer features automatically without a client
side code change. For example, consider the introduction of listkeys light. The
application just calls list as usual, but the client interprets that call to go
to either the original listkeys or lightweight listkeys based on the OM version.
* Outside of S3g, clients cannot compare their {{OzoneManagerVersion}} enum
with the version the OM returned to determine whether OM is finalized. If the
client is newer than the OM, even a finalized OM will return a lower version.
I recommend the simplest solution that handles this automatically, which is to
have clients refresh their OM service info periodically. Every 10 minutes seems
reasonable to me. Later we could end up with other pieces of information that
can change on the server, so it could be generally useful to expire the
client's cached service info periodically. If this change doesn't make it in
time for the next release we can document a rolling restart of S3 gateways as
an additional step after finalization.
> S3 Gateway does not pick up the new OM version after ZDU finalization
> ----------------------------------------------------------------------
>
> Key: HDDS-17140
> URL: https://issues.apache.org/jira/browse/HDDS-17140
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Chung-En Lee
> Assignee: Andy Huang
> Priority: Major
>
> The ZDU design doc ("Order of Operations During the Upgrade", step 5) says to
> rolling-restart S3 Gateway and other client processes on the new software
> before the cluster is finalized. At that point OM is still pre-finalized, so
> it reports the old version to clients.
> S3 Gateway learns the OM version only once, when it handles its first request
> after starting, and keeps it for the life of the process. In a ZDU rolling
> restart S3 Gateway keeps serving traffic, so it captures the old version
> before finalization. After the admin finalizes the cluster, OM reports the
> new version, but S3 Gateway keeps the old one. Any new feature that depends
> on the finalized OM version stays disabled in S3 Gateway until it is
> restarted again.
> Proposed fix (options):
> 1. Change the documented upgrade order so S3 Gateway is restarted after
> finalization.
> 2. Keep the current order, and document that S3 Gateway must be restarted
> again after finalization.
> 3. Make S3 Gateway aware OM version changes (e.g. after finalization) and
> re-initialize its OzoneClient automatically, so it picks up the latest OM
> version without a manual restart.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
