Toan Le created HDFS-17881:
------------------------------
Summary: Hadoop file system can not browse the folder that has
children folder with colon in Windows
Key: HDFS-17881
URL: https://issues.apache.org/jira/browse/HDFS-17881
Project: Hadoop HDFS
Issue Type: Bug
Components: fs/azure
Affects Versions: 3.4.2
Reporter: Toan Le
Attachments: Screenshot 2026-02-11 at 11.40.36.png,
test_hadoop_azure.zip
Hi guys,
I am facing an issue when browsing the children's folder name with a colon in
ADLS.
Here is my case. My root folder in ADLS contains a folder with a name that
includes a colon.
When I used the Hadoop file system. I got this error on Windows:
{code:java}
java -cp ".;./*" AbfsListStatusTest.java [main] WARN
org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable [main] ERROR
org.apache.hadoop.fs.azurebfs.services.AbfsClient - Unable to deserialize list
results for Uri abfss://<containerName>@<storeAccountName>.dfs.core.windows.net
java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal
character in scheme name at index 0: \FestSpecial: at
org.apache.hadoop.fs.Path.initialize(Path.java:264) at
org.apache.hadoop.fs.Path.<init>(Path.java:222) at
org.apache.hadoop.fs.azurebfs.services.AbfsClient.getVersionedFileStatusFromEntry(AbfsClient.java:1846)
at
org.apache.hadoop.fs.azurebfs.services.AbfsDfsClient.parseListPathResults(AbfsDfsClient.java:1373)
at
org.apache.hadoop.fs.azurebfs.services.AbfsDfsClient.listPath(AbfsDfsClient.java:347)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1278)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1242)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1224)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.listStatus(AzureBlobFileSystem.java:713)
at AbfsListStatusOAuthTest.main(AbfsListStatusTest.java:45) at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at jdk.compiler/com.sun.tools.javac.launcher.Main.execute(Main.java:484)
at jdk.compiler/com.sun.tools.javac.launcher.Main.run(Main.java:208)
at jdk.compiler/com.sun.tools.javac.launcher.Main.main(Main.java:135) Caused
by: java.net.URISyntaxException: Illegal character in scheme name at index 0:
\FestSpecial:
at java.base/java.net.URI$Parser.fail(URI.java:2995)
at java.base/java.net.URI$Parser.checkChars(URI.java:3166)
at java.base/java.net.URI$Parser.checkChar(URI.java:3176)
at java.base/java.net.URI$Parser.parse(URI.java:3191)
at java.base/java.net.URI.<init>(URI.java:807)
at org.apache.hadoop.fs.Path.initialize(Path.java:261)
... 14 more Exception in thread "main" Status code: -1 error code: null error
message: Parsing of Json List Response Failed in
DfsClient.java.lang.IllegalArgumentException: java.net.URISyntaxException:
Illegal character in scheme name at index 0: \FestSpecial: at
org.apache.hadoop.fs.azurebfs.services.AbfsDfsClient.parseListPathResults(AbfsDfsClient.java:1390)
at
org.apache.hadoop.fs.azurebfs.services.AbfsDfsClient.listPath(AbfsDfsClient.java:347)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1278)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1242)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1224)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.listStatus(AzureBlobFileSystem.java:713)
at AbfsListStatusOAuthTest.main(AbfsListStatusTest.java:45) Caused by:
java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal
character in scheme name at index 0: \FestSpecial: at
org.apache.hadoop.fs.Path.initialize(Path.java:264) at
org.apache.hadoop.fs.Path.<init>(Path.java:222) at
org.apache.hadoop.fs.azurebfs.services.AbfsClient.getVersionedFileStatusFromEntry(AbfsClient.java:1846)
at
org.apache.hadoop.fs.azurebfs.services.AbfsDfsClient.parseListPathResults(AbfsDfsClient.java:1373)
at
org.apache.hadoop.fs.azurebfs.services.AbfsDfsClient.listPath(AbfsDfsClient.java:347)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1278)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1242)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.listStatus(AzureBlobFileSystemStore.java:1224)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.listStatus(AzureBlobFileSystem.java:713)
at AbfsListStatusOAuthTest.main(AbfsListStatusTest.java:45) at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at jdk.compiler/com.sun.tools.javac.launcher.Main.execute(Main.java:484)
at jdk.compiler/com.sun.tools.javac.launcher.Main.run(Main.java:208)
at jdk.compiler/com.sun.tools.javac.launcher.Main.main(Main.java:135) Caused
by: java.net.URISyntaxException: Illegal character in scheme name at index 0:
\FestSpecial: at java.base/java.net.URI$Parser.fail(URI.java:2995)
at java.base/java.net.URI$Parser.checkChars(URI.java:3166) at
java.base/java.net.URI$Parser.checkChar(URI.java:3176) at
java.base/java.net.URI$Parser.parse(URI.java:3191) at
java.base/java.net.URI.<init>(URI.java:807) at
org.apache.hadoop.fs.Path.initialize(Path.java:261) ... 14 more
{code}
I tested my script, and it worked on Linux.
Could you please help me check it?
I compressed all my scripts ({_}AbfsListStatusTest.java{_}) and library into
the ticket ({_}test_hadoop_azure.zip{_}).
Here is the _AbfsListStatusTest.java:_
{code:java}
{code}
_import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import java.net.URI;
public class AbfsListStatusOAuthTest \{
public static void main(String[] args) throws Exception {
// Inputs (use env vars or hardcode)
String accountName = System.getenv("ADLS_STORAGE_ACCOUNT_NAME");
String containerName = System.getenv("ADLS_CONTAINER_NAME");
String tenantId = System.getenv("ADLS_TENANT_ID");
String clientId = System.getenv("ADLS_CLIENT_ID");
String clientSecret = System.getenv("ADLS_CLIENT_SECRET");
String dirPath = (args.length > 0) ? args[0] : "/";
// Use abfss for OAuth
String fsUri = String.format("abfss://%s@%s.dfs.core.windows.net",
containerName, accountName);
String accountFqdn = accountName + ".dfs.core.windows.net";
Configuration conf = new Configuration(false);
// Ensure ABFS impls are known
conf.set("fs.abfss.impl",
"org.apache.hadoop.fs.azurebfs.SecureAzureBlobFileSystem");
conf.set("fs.abfs.impl",
"org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem");
// --- OAuth configs (scoped to the account host) ---
conf.set("fs.azure.account.auth.type." + accountFqdn, "OAuth");
conf.set("fs.azure.account.oauth.provider.type." + accountFqdn,
"org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider");
conf.set("fs.azure.account.oauth2.client.id." + accountFqdn, clientId);
conf.set("fs.azure.account.oauth2.client.secret." + accountFqdn,
clientSecret);
// Token endpoint (try v1 first — Hadoop ABFS commonly uses this)
conf.set("fs.azure.account.oauth2.client.endpoint." + accountFqdn,
"https://login.microsoftonline.com/"; + tenantId +
"/oauth2/token");
// Create FS and call listStatus
FileSystem fs = FileSystem.get(new URI(fsUri), conf);
FileStatus[] statuses = fs.listStatus(new Path(dirPath));
System.out.println("Found " + statuses.length + " entries under " +
dirPath);
for (FileStatus st : statuses) {
System.out.printf("%s isDir=%s len=%d%n",
st.getPath(), st.isDirectory(), st.getLen());
}
fs.close();
}
}_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
