Copilot commented on code in PR #13455:
URL: https://github.com/apache/gravitino/pull/13455#discussion_r4078342785
##########
clients/filesystem-hadoop3/src/main/java/org/apache/gravitino/filesystem/hadoop/BaseGVFSOperations.java:
##########
@@ -449,6 +449,25 @@ public abstract FSDataOutputStream append(Path gvfsPath,
int bufferSize, Progres
*/
public abstract Token<?>[] addDelegationTokens(String renewer, Credentials
credentials);
+ /**
+ * Resolve the given fileset path so that its actual file system is created
before delegation
+ * tokens are collected.
+ *
+ * <p>The file system cache is only populated when a file is read, but Spark
collects tokens
+ * before the job starts. Without this step no token is returned for a
fileset that lives outside
+ * the default file system.
+ *
+ * @param filesetPath the virtual path this file system was initialized with.
+ */
+ public void prepareDelegationTokens(Path filesetPath) {
+ try {
+ getActualFileSystem(filesetPath, null);
Review Comment:
This resolves the default location unconditionally, but normal GVFS
operations use `currentLocationName()` (for example,
`DefaultGVFSOperations.open`), which may select a non-default location. For a
multi-location fileset whose selected location has a different filesystem
authority, token collection caches only the default filesystem while subsequent
reads use the selected one, so executors still lack that location's token.
Resolve with the same current location used by data operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
