[
https://issues.apache.org/jira/browse/IMPALA-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116466#comment-18116466
]
Michael Smith commented on IMPALA-15368:
----------------------------------------
Do we know what change caused this?
> parquet-delta-length-byte-array-decoder-test fails in ASAN builds
> -----------------------------------------------------------------
>
> Key: IMPALA-15368
> URL: https://issues.apache.org/jira/browse/IMPALA-15368
> Project: IMPALA
> Issue Type: Bug
> Components: Test
> Reporter: Dániel Gábor Vankó
> Assignee: Dániel Gábor Vankó
> Priority: Major
> Labels: broken-build
>
> We have a heap-buffer-overflow in
> ParquetDeltaLengthByteArrayDecoderTest.EmptyPage test.
> h2. Error Message
> {code:java}
> Build command failed:
> /data0/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/cmake-3.31.11/bin/ctest
> --force-new-ctest-process
> {code}
> h2. Standard Output
> {code:java}
> Test project /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala
> Start 24: parquet-delta-length-byte-array-decoder-test
> 24/167 Test #24: parquet-delta-length-byte-array-decoder-test ...***Failed
> 2.65 sec
> INFO: Stack trace signal handler initialized with signal 44 (SIGRTMIN+10).
> Access thread stacks via the /stacks web UI (DO NOT manually send this
> signal).
> OpenJDK VM warning: the use of signal() and sigset() for signal chaining was
> deprecated in version 16.0 and will be removed in a future release. Use
> sigaction() instead.
> Picked up JAVA_TOOL_OPTIONS:
> -javaagent:/var/lib/jenkins/.m2/repository/com/github/jbellis/jamm/0.4.0/jamm-0.4.0.jar
> --add-opens=java.base/java.lang=ALL-UNNAMED
> --add-opens=java.base/java.nio=ALL-UNNAMED
> --add-opens=java.base/java.util.regex=ALL-UNNAMED
> --add-opens=java.base/sun.nio.ch=ALL-UNNAMED -Dsun.java.command=unifiedbetests
> 26/09/09 21:15:17 WARN fs.FileSystem: Cannot load filesystem
> java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem:
> Provider org.apache.hadoop.hive.ql.io.NullScanFileSystem not found
> at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
> at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1219)
> at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1228)
> at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:3522)
> 26/09/09 21:15:17 WARN fs.FileSystem: Cannot load filesystem
> java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem:
> Provider org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem not found
> at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
> at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1219)
> at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1228)
> at
> java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
> at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
> at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
> at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:3522)
> seed = 1789013717
> Note: Google Test filter = ParquetDeltaLengthByteArrayDecoderTest.*
> [==========] Running 24 tests from 1 test suite.
> [----------] Global test environment set-up.
> [----------] 24 tests from ParquetDeltaLengthByteArrayDecoderTest
> [ RUN ] ParquetDeltaLengthByteArrayDecoderTest.BasicRoundTrip
> [ OK ] ParquetDeltaLengthByteArrayDecoderTest.BasicRoundTrip (0 ms)
> [ RUN ] ParquetDeltaLengthByteArrayDecoderTest.EmptyPage
> =================================================================
> ==1104662==ERROR: AddressSanitizer: heap-buffer-overflow on address
> 0x60200029ffd1 at pc 0x000002f78c2c bp 0x7fffb4272bb0 sp 0x7fffb4272360
> WRITE of size 1 at 0x60200029ffd1 thread T0
> 26/09/09 21:15:17 INFO util.JvmPauseMonitor: Starting JVM pause monitor
> #0 0x2f78c2b in __asan_memcpy
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2f78c2b)
> #1 0x30d8fa6 in bool impala::BitWriter::PutAligned<unsigned
> char>(unsigned char, int)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util/bit-stream-utils.inline.h:84:3
> #2 0x30d8d14 in bool impala::BitWriter::PutUleb128<unsigned int>(unsigned
> int)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util/bit-stream-utils.inline.h:101:13
> #3 0x69b3ce0 in impala::ParquetDeltaEncoder<int>::WriteHeader()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-encoder.cc:231:31
> #4 0x69b3371 in impala::ParquetDeltaEncoder<int>::FinalizePage()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-encoder.cc:127:31
> #5 0x31d66e3 in
> impala::BuildPage(std::vector<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > > > const&, std::vector<unsigned char,
> std::allocator<unsigned char> >*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-length-byte-array-decoder-test.cc:49:23
> #6 0x31d7314 in
> impala::ParquetDeltaLengthByteArrayDecoderTest_EmptyPage_Test::TestBody()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-length-byte-array-decoder-test.cc:85:3
> #7 0xa60ea8c in void
> testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
> #8 0xa60ea8c in void
> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
> #9 0xa5f383d in testing::Test::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2687:50
> #10 0xa5f383d in testing::Test::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2677:6
> #11 0xa5f39f4 in testing::TestInfo::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2836:14
> #12 0xa5f3bb0 in testing::TestSuite::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:3015:33
> #13 0xa5f3bb0 in testing::TestSuite::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2968:6
> #14 0xa60694e in testing::internal::UnitTestImpl::RunAllTests()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:5920:47
> #15 0xa5f3dc4 in bool
> testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
> bool>(testing::internal::UnitTestImpl*, bool
> (testing::internal::UnitTestImpl::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
> #16 0xa5f3dc4 in bool
> testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
> bool>(testing::internal::UnitTestImpl*, bool
> (testing::internal::UnitTestImpl::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
> #17 0xa5f3dc4 in testing::UnitTest::Run()
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:5484:55
> #18 0x2fc0241 in main
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10
> #19 0x7faa32c2c864 in __libc_start_main (/lib64/libc.so.6+0x3a864)
> #20 0x2ebc11d in _start
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2ebc11d)
> 0x60200029ffd1 is located 0 bytes to the right of 1-byte region
> [0x60200029ffd0,0x60200029ffd1)
> allocated by thread T0 here:
> #0 0x2fbce5f in operator new(unsigned long)
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2fbce5f)
> #1 0x30b0b01 in std::_Vector_base<unsigned char, std::allocator<unsigned
> char> >::_M_create_storage(unsigned long)
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:361:33
> #2 0x30b07df in std::_Vector_base<unsigned char, std::allocator<unsigned
> char> >::_Vector_base(unsigned long, std::allocator<unsigned char> const&)
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:305:9
> #3 0x30a706b in std::vector<unsigned char, std::allocator<unsigned char>
> >::vector(unsigned long, std::allocator<unsigned char> const&)
> /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:511:9
> #4 0x31d6504 in
> impala::BuildPage(std::vector<std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >,
> std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > > > const&, std::vector<unsigned char,
> std::allocator<unsigned char> >*)
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-length-byte-array-decoder-test.cc:46:19
> #5 0x31d7314 in
> impala::ParquetDeltaLengthByteArrayDecoderTest_EmptyPage_Test::TestBody()
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/exec/parquet/parquet-delta-length-byte-array-decoder-test.cc:85:3
> #6 0xa60ea8c in void
> testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2612:27
> #7 0xa60ea8c in void
> testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,
> void>(testing::Test*, void (testing::Test::*)(), char const*)
> /mnt/source/googletest/googletest-1.14.0/googletest/src/gtest.cc:2648:52
> SUMMARY: AddressSanitizer: heap-buffer-overflow
> (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2f78c2b)
> in __asan_memcpy
> Shadow bytes around the buggy address:
> 0x0c048004bfa0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
> 0x0c048004bfb0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
> 0x0c048004bfc0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
> 0x0c048004bfd0: fa fa 00 00 fa fa 00 fa fa fa fd fd fa fa fd fa
> 0x0c048004bfe0: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fd
> =>0x0c048004bff0: fa fa fd fa fa fa 00 00 fa fa[01]fa fa fa fa fa
> 0x0c048004c000: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
> 0x0c048004c010: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
> 0x0c048004c020: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
> 0x0c048004c030: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
> 0x0c048004c040: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user: f7
> Container overflow: fc
> Array cookie: ac
> Intra object redzone: bb
> ASan internal: fe
> Left alloca redzone: ca
> Right alloca redzone: cb
> ==1104662==ABORTING
> Traceback (most recent call last):
> File
> "/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/bin/junitxml_prune_notrun.py",
> line 71, in <module>
> if __name__ == "__main__": main()
> File
> "/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/bin/junitxml_prune_notrun.py",
> line 68, in main
> junitxml_prune_notrun(options.filename)
> File
> "/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/bin/junitxml_prune_notrun.py",
> line 31, in junitxml_prune_notrun
> root = tree.parse(junitxml_filename)
> File "/usr/lib64/python3.8/xml/etree/ElementTree.py", line 584, in parse
> source = open(source, "rb")
> FileNotFoundError: [Errno 2] No such file or directory:
> '/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/be_tests/parquet-delta-length-byte-array-decoder-test.xml'
> 99% tests passed, 1 tests failed out of 167
> Total Test time (real) = 2676.12 sec
> The following tests FAILED:
> 24 - parquet-delta-length-byte-array-decoder-test (Failed)
> {code}
> h2. Standard Error
> {code:java}
> Errors while running CTest
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
