Dániel Gábor Vankó created IMPALA-15368:
-------------------------------------------
Summary: parquet-delta-length-byte-array-decoder-test fails in
ASAN builds
Key: IMPALA-15368
URL: https://issues.apache.org/jira/browse/IMPALA-15368
Project: IMPALA
Issue Type: Bug
Components: Test
Reporter: Dániel Gábor Vankó
Assignee: Dániel Gábor Vankó
We have a heap-buffer-overflow in
ParquetDeltaLengthByteArrayDecoderTest.EmptyPage test.
h2. Error Message
{code:java}
Build command failed:
/data0/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/cmake-3.31.11/bin/ctest
--force-new-ctest-process
{code}
h2. Standard Output
{code:java}
Test project /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala
Start 24: parquet-delta-length-byte-array-decoder-test
24/167 Test #24: parquet-delta-length-byte-array-decoder-test ...***Failed
2.65 sec
INFO: Stack trace signal handler initialized with signal 44 (SIGRTMIN+10).
Access thread stacks via the /stacks web UI (DO NOT manually send this signal).
OpenJDK VM warning: the use of signal() and sigset() for signal chaining was
deprecated in version 16.0 and will be removed in a future release. Use
sigaction() instead.
Picked up JAVA_TOOL_OPTIONS:
-javaagent:/var/lib/jenkins/.m2/repository/com/github/jbellis/jamm/0.4.0/jamm-0.4.0.jar
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util.regex=ALL-UNNAMED
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED -Dsun.java.command=unifiedbetests
26/09/09 21:15:17 WARN fs.FileSystem: Cannot load filesystem
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hive.ql.io.NullScanFileSystem not found
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1219)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1228)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:3522)
26/09/09 21:15:17 WARN fs.FileSystem: Cannot load filesystem
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem not found
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1219)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1228)
at
java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:3522)
seed = 1789013717
Note: Google Test filter = ParquetDeltaLengthByteArrayDecoderTest.*
[==========] Running 24 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 24 tests from ParquetDeltaLengthByteArrayDecoderTest
[ RUN ] ParquetDeltaLengthByteArrayDecoderTest.BasicRoundTrip
[ OK ] ParquetDeltaLengthByteArrayDecoderTest.BasicRoundTrip (0 ms)
[ RUN ] ParquetDeltaLengthByteArrayDecoderTest.EmptyPage
=================================================================
==1104662==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x60200029ffd1 at pc 0x000002f78c2c bp 0x7fffb4272bb0 sp 0x7fffb4272360
WRITE of size 1 at 0x60200029ffd1 thread T0
26/09/09 21:15:17 INFO util.JvmPauseMonitor: Starting JVM pause monitor
#0 0x2f78c2b in __asan_memcpy
(/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2f78c2b)
#1 0x30d8fa6 in bool impala::BitWriter::PutAligned<unsigned char>(unsigned
char, int)
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util/bit-stream-utils.inline.h:84:3
#2 0x30d8d14 in bool impala::BitWriter::PutUleb128<unsigned int>(unsigned
int)
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util/bit-stream-utils.inline.h:101:13
#3 0x69b3ce0 in impala::ParquetDeltaEncoder<int>::WriteHeader()
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-encoder.cc:231:31
#4 0x69b3371 in impala::ParquetDeltaEncoder<int>::FinalizePage()
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-encoder.cc:127:31
#5 0x31d66e3 in
impala::BuildPage(std::vector<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > > > const&, std::vector<unsigned char,
std::allocator<unsigned char> >*)
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-length-byte-array-decoder-test.cc:49:23
#6 0x31d7314 in
impala::ParquetDeltaLengthByteArrayDecoderTest_EmptyPage_Test::TestBody()
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-length-byte-array-decoder-test.cc:85:3
#7 0xa60ea8c in void
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
void>(testing::Test*, void (testing::Test::*)(), char const*)
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
#8 0xa60ea8c in void
testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
void>(testing::Test*, void (testing::Test::*)(), char const*)
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
#9 0xa5f383d in testing::Test::Run()
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2687:50
#10 0xa5f383d in testing::Test::Run()
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2677:6
#11 0xa5f39f4 in testing::TestInfo::Run()
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2836:14
#12 0xa5f3bb0 in testing::TestSuite::Run()
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:3015:33
#13 0xa5f3bb0 in testing::TestSuite::Run()
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2968:6
#14 0xa60694e in testing::internal::UnitTestImpl::RunAllTests()
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:5920:47
#15 0xa5f3dc4 in bool
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool>(testing::internal::UnitTestImpl*, bool
(testing::internal::UnitTestImpl::*)(), char const*)
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
#16 0xa5f3dc4 in bool
testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
bool>(testing::internal::UnitTestImpl*, bool
(testing::internal::UnitTestImpl::*)(), char const*)
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
#17 0xa5f3dc4 in testing::UnitTest::Run()
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:5484:55
#18 0x2fc0241 in main
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10
#19 0x7faa32c2c864 in __libc_start_main (/lib64/libc.so.6+0x3a864)
#20 0x2ebc11d in _start
(/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2ebc11d)
0x60200029ffd1 is located 0 bytes to the right of 1-byte region
[0x60200029ffd0,0x60200029ffd1)
allocated by thread T0 here:
#0 0x2fbce5f in operator new(unsigned long)
(/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2fbce5f)
#1 0x30b0b01 in std::_Vector_base<unsigned char, std::allocator<unsigned
char> >::_M_create_storage(unsigned long)
/data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:361:33
#2 0x30b07df in std::_Vector_base<unsigned char, std::allocator<unsigned
char> >::_Vector_base(unsigned long, std::allocator<unsigned char> const&)
/data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:305:9
#3 0x30a706b in std::vector<unsigned char, std::allocator<unsigned char>
>::vector(unsigned long, std::allocator<unsigned char> const&)
/data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:511:9
#4 0x31d6504 in
impala::BuildPage(std::vector<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > > > const&, std::vector<unsigned char,
std::allocator<unsigned char> >*)
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-length-byte-array-decoder-test.cc:46:19
#5 0x31d7314 in
impala::ParquetDeltaLengthByteArrayDecoderTest_EmptyPage_Test::TestBody()
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-length-byte-array-decoder-test.cc:85:3
#6 0xa60ea8c in void
testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
void>(testing::Test*, void (testing::Test::*)(), char const*)
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
#7 0xa60ea8c in void
testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
void>(testing::Test*, void (testing::Test::*)(), char const*)
/mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
SUMMARY: AddressSanitizer: heap-buffer-overflow
(/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2f78c2b)
in __asan_memcpy
Shadow bytes around the buggy address:
0x0c048004bfa0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
0x0c048004bfb0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
0x0c048004bfc0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
0x0c048004bfd0: fa fa 00 00 fa fa 00 fa fa fa fd fd fa fa fd fa
0x0c048004bfe0: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fd
=>0x0c048004bff0: fa fa fd fa fa fa 00 00 fa fa[01]fa fa fa fa fa
0x0c048004c000: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
0x0c048004c010: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
0x0c048004c020: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
0x0c048004c030: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
0x0c048004c040: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==1104662==ABORTING
Traceback (most recent call last):
File
"/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/bin/junitxml_prune_notrun.py",
line 71, in <module>
if __name__ == "__main__": main()
File
"/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/bin/junitxml_prune_notrun.py",
line 68, in main
junitxml_prune_notrun(options.filename)
File
"/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/bin/junitxml_prune_notrun.py",
line 31, in junitxml_prune_notrun
root = tree.parse(junitxml_filename)
File "/usr/lib64/python3.8/xml/etree/ElementTree.py", line 584, in parse
source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory:
'/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/be_tests/parquet-delta-length-byte-array-decoder-test.xml'
99% tests passed, 1 tests failed out of 167
Total Test time (real) = 2676.12 sec
The following tests FAILED:
24 - parquet-delta-length-byte-array-decoder-test (Failed)
{code}
h2. Standard Error
{code:java}
Errors while running CTest
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
