[
https://issues.apache.org/jira/browse/HDFS-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri resolved HDFS-15944.
--------------------------------
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Prevent truncation by snprintf
> ------------------------------
>
> Key: HDFS-15944
> URL: https://issues.apache.org/jira/browse/HDFS-15944
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fuse-dfs, libhdfs
> Affects Versions: 3.4.0
> Reporter: Gautham Banasandra
> Assignee: Gautham Banasandra
> Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> There are some areas of code in libhdfs and fuse-dfs components where the
> destination buffer is smaller than the source that's trying to write. This
> would cause truncation. Thus we need to ensure that the source that's being
> written doesn't exceed the destination buffer size.
> The following warnings are reported for this issue -
> {code}
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:
> In function ‘doTestHdfsOperations.isra.0’:
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:226:39:
> warning: ‘/many_files_’ directive output may be truncated writing 12 bytes
> into a region of size between 1 and 4096 [-Wformat-truncation=]
> 226 | snprintf(filename, PATH_MAX, "%s/many_files_%d", listDirTest,
> nFile);
> | ^~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:226:36:
> note: directive argument in the range [0, 9999]
> 226 | snprintf(filename, PATH_MAX, "%s/many_files_%d", listDirTest,
> nFile);
> | ^~~~~~~~~~~~~~~~~~
> In file included from /usr/include/stdio.h:867,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c:19:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note:
> ‘__builtin___snprintf_chk’ output between 14 and 4112 bytes into a
> destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> {code}
> {code}
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:255:33:
> warning: ‘/a’ directive output may be truncated writing 2 bytes into a
> region of size between 1 and 4096 [-Wformat-truncation=]
> 255 | snprintf(tmp, sizeof(tmp), "%s/a", base);
> | ^~
> In file included from /usr/include/stdio.h:867,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note:
> ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination
> of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:263:33:
> warning: ‘/b’ directive output may be truncated writing 2 bytes into a
> region of size between 1 and 4096 [-Wformat-truncation=]
> 263 | snprintf(tmp, sizeof(tmp), "%s/b", base);
> | ^~
> In file included from /usr/include/stdio.h:867,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note:
> ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination
> of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:271:33:
> warning: ‘/a’ directive output may be truncated writing 2 bytes into a
> region of size between 1 and 4096 [-Wformat-truncation=]
> 271 | snprintf(src, sizeof(src), "%s/a", base);
> | ^~
> In file included from /usr/include/stdio.h:867,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note:
> ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination
> of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:272:33:
> warning: ‘/c’ directive output may be truncated writing 2 bytes into a
> region of size between 1 and 4096 [-Wformat-truncation=]
> 272 | snprintf(dst, sizeof(dst), "%s/c", base);
> | ^~
> In file included from /usr/include/stdio.h:867,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note:
> ‘__builtin___snprintf_chk’ output between 3 and 4098 bytes into a destination
> of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:297:35:
> warning: ‘/b/’ directive output may be truncated writing 3 bytes into a
> region of size between 1 and 4096 [-Wformat-truncation=]
> 297 | snprintf(tmp, sizeof(tmp), "%s/b/%d", base, i);
> | ^~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:297:32:
> note: directive argument in the range [0, 2]
> 297 | snprintf(tmp, sizeof(tmp), "%s/b/%d", base, i);
> | ^~~~~~~~~
> In file included from /usr/include/stdio.h:867,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note:
> ‘__builtin___snprintf_chk’ output between 5 and 4100 bytes into a destination
> of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:187:35:
> warning: ‘/trunc.txt’ directive output may be truncated writing 10 bytes
> into a region of size between 1 and 4096 [-Wformat-truncation=]
> 187 | snprintf(path, sizeof(path), "%s/trunc.txt", base);
> | ^~~~~~~~~~
> In file included from /usr/include/stdio.h:867,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/expect.h:23,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/test/fuse_workload.c:22:
> /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:10: note:
> ‘__builtin___snprintf_chk’ output between 11 and 4106 bytes into a
> destination of size 4096
> 67 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 68 | __bos (__s), __fmt, __va_arg_pack ());
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
