[
https://issues.apache.org/jira/browse/IMPALA-14181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17985721#comment-17985721
]
Joe McDonnell commented on IMPALA-14181:
----------------------------------------
Stack:
{noformat}
6 impalad!google::LogMessageFatal::~LogMessageFatal() [logging.cc : 2579 +
0x5]
7 impalad!bool
impala::Sorter::Run::ConvertValueOffsetsToPtrs<impala::CollectionValue>(impala::Tuple*,
unsigned char*, std::vector<impala::SlotDescriptor*,
std::allocator<impala::SlotDescriptor*> > const&) [sorter.cc : 829 + 0x8]
8
impalad!impala::Sorter::Run::ConvertCollectionValueOffsetsToPtrs(impala::Tuple*,
impala::TupleDescriptor const&, unsigned char*) [sorter.cc : 873 + 0x8]
9 impalad!impala::Sorter::Run::ConvertOffsetsToPtrs(impala::Tuple*,
impala::TupleDescriptor const&) [sorter.cc : 762 + 0x11]
10 impalad!impala::Status
impala::Sorter::Run::GetNext<true>(impala::RowBatch*, bool*) [sorter.cc : 557 +
0xb]
11 impalad!impala::Sorter::Run::GetNextBatch(impala::RowBatch**) [sorter.cc :
487 + 0x10]
12
impalad!boost::detail::function::function_obj_invoker1<boost::_bi::bind_t<impala::Status,
boost::_mfi::mf1<impala::Status, impala::Sorter::Run, impala::RowBatch**>,
boost::_bi::list2<boost::_bi::value<impala::Sorter::Run*>, boost::arg<1> > >,
impala::Status,
impala::RowBatch**>::invoke(boost::detail::function::function_buffer&,
impala::RowBatch**) [mem_fn_template.hpp : 165 + 0xc]
13 impalad!impala::SortedRunMerger::SortedRunWrapper::Init(bool*)
[function_template.hpp : 764 + 0x7]
14
impalad!impala::SortedRunMerger::Prepare(std::vector<boost::function<impala::Status
(impala::RowBatch**)>, std::allocator<boost::function<impala::Status
(impala::RowBatch**)> > > const&) [sorted-run-merger.cc : 105 + 0x13]
15 impalad!impala::Sorter::CreateMerger(int, bool) [sorter.cc : 1586 + 0x10]
16 impalad!impala::Sorter::MergeIntermediateRuns() [sorter.cc : 1539 + 0x11]
17 impalad!impala::Sorter::InputDone() [sorter.cc : 1349 + 0xb]
18 impalad!impala::SortNode::SortInput(impala::RuntimeState*) [sort-node.cc :
245 + 0x9]
19 impalad!impala::SortNode::Open(impala::RuntimeState*) [sort-node.cc : 142 +
0xf]{noformat}
> DCHECK in Sorter::Run::ConvertValueOffsetsToPtrs() with low memory
> ------------------------------------------------------------------
>
> Key: IMPALA-14181
> URL: https://issues.apache.org/jira/browse/IMPALA-14181
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Priority: Major
>
> When testing with tuple caching, queries start to use more memory. I hit this
> assert while running test_sort.py's TestArraySort:
> {noformat}
> F20250623 14:59:49.492782 1002052 sorter.cc:829]
> d84fbaa341de8fbb:027dfece00000000] Check failed: page_offset == 0 (2311 vs.
> 0) {noformat}
> From here:
> {noformat}
> if (page_index > var_len_pages_index_) {
> // We've reached the page boundary for the current var-len page.
> // This tuple will be returned in the next call to GetNext().
> DCHECK_GE(page_index, 0);
> DCHECK_LE(page_index, var_len_pages_.size());
> DCHECK_EQ(page_index, var_len_pages_index_ + 1);
> DCHECK_EQ(page_offset, 0); <-------------- HERE
> // The data is the first thing in the next
> page.
> // This must be the first slot with var len
> data for the
> // tuple. Var len data for tuple shouldn't
> be split
> // across blocks.
> DCHECK(AllPrevSlotsAreNullsOrSmall<ValueType>(tuple, slots, idx));
> return false;
> }{noformat}
> This is easy to reproduce by using a slightly tighter memory value. On my
> machine, this works:
> {noformat}
> set max_sort_run_size=2;
> set num_nodes=1;
> -- ordinarily buffer_pool_limit=44m, but use a slightly tighter value
> set buffer_pool_limit=41m;
> select string_col, int_array, double_map, string_array, mixed from
> functional_parquet.arrays_big order by string_col;{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
