[
https://issues.apache.org/jira/browse/IMPALA-13770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932641#comment-17932641
]
ASF subversion and git services commented on IMPALA-13770:
----------------------------------------------------------
Commit 24179d46cce2a1ff063f9ae37f0291f2dd23d8ed in impala's branch
refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=24179d46c ]
IMPALA-13793: Always close expressions
The Expr class has a Close() method, which is currently only used for
releasing 'cache_entry_', which is used for UDF or UDAF functions. If
Close() is not called for an Expr that uses 'cache_entry_', a DCHECK
fails in the destructor of Expr. However, if an Expr does not use
'cache_entry_' is not closed, nothing happens.
Classes holding Expr objects should always close
expressions because they may use 'cache_entry_'. However, if tests fail
to include cases where 'cache_entry_' is used, bugs may not be detected:
one such case was IMPALA-13770.
In debug builds, this change adds a new field, 'closed_', to the Expr
class, which is set to true in Close(). A DCHECK in the destructor fires
if Close() has not been called, regardless of whether 'cache_entry_' is
used. This makes it easier to detect cases where expressions are not
closed.
To avoid increasing the size of expressions, the 'closed_' field is not
added in release mode.
Existing cases where expressions are not closed are fixed in this
change.
Testing:
- tests are updated to close expressions
Change-Id: I9784652f5e135f1f8cd7f0fa6df471e64d27c477
Reviewed-on: http://gerrit.cloudera.org:8080/22528
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Updating Iceberg tables with UDFs crashes Impala
> ------------------------------------------------
>
> Key: IMPALA-13770
> URL: https://issues.apache.org/jira/browse/IMPALA-13770
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Noemi Pap-Takacs
> Assignee: Daniel Becker
> Priority: Major
> Labels: impala-iceberg
>
> When updating an Iceberg table setting the column to a value defined by a
> UDF, Impala crashes during execution on the Backend with the following stack
> trace:
> {noformat}
> #7 0x0000000005c89709 in google::LogMessageFatal::~LogMessageFatal() ()
> #8 0x00000000041caa55 in impala::Expr::~Expr (this=0x148bfc00,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/exprs/expr.cc:47
> #9 0x00000000041cf4c6 in impala::ScalarExpr::~ScalarExpr (this=0x148bfc00,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/exprs/scalar-expr.h:140
> #10 0x00000000041e1bd0 in impala::ScalarFnCall::~ScalarFnCall
> (this=0x148bfc00, __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/exprs/scalar-fn-call.h:68
> #11 0x00000000041e1bec in impala::ScalarFnCall::~ScalarFnCall
> (this=0x148bfc00, __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/exprs/scalar-fn-call.h:68
> #12 0x00000000041e241e in
> impala::ObjectPool::Add<impala::ScalarFnCall>(impala::ScalarFnCall*)::{lambda(void*)#1}::operator()(void*)
> const (this=0x0, obj=0x148bfc00) at
> /home/user/Impala/be/src/common/object-pool.h:41
> #13 0x00000000041e243e in
> impala::ObjectPool::Add<impala::ScalarFnCall>(impala::ScalarFnCall*)::{lambda(void*)#1}::_FUN(void*)
> () at /home/user/Impala/be/src/common/object-pool.h:41
> #14 0x00000000023fd213 in impala::ObjectPool::Clear (this=0x153289a0) at
> /home/user/Impala/be/src/common/object-pool.h:47
> #15 0x00000000023fd17c in impala::ObjectPool::~ObjectPool (this=0x153289a0,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/common/object-pool.h:34
> #16 0x0000000002b31f80 in impala::FragmentState::~FragmentState
> (this=0x153289a0, __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/runtime/fragment-state.cc:192
> #17 0x0000000002b34000 in
> impala::ObjectPool::Add<impala::FragmentState>(impala::FragmentState*)::{lambda(void*)#1}::operator()(void*)
> const (this=0x0, obj=0x153289a0) at
> /home/user/Impala/be/src/common/object-pool.h:41
> #18 0x0000000002b34031 in
> impala::ObjectPool::Add<impala::FragmentState>(impala::FragmentState*)::{lambda(void*)#1}::_FUN(void*)
> () at /home/user/Impala/be/src/common/object-pool.h:41
> #19 0x00000000023fd213 in impala::ObjectPool::Clear (this=0x13c0aab8) at
> /home/user/Impala/be/src/common/object-pool.h:47
> #20 0x00000000023fd17c in impala::ObjectPool::~ObjectPool (this=0x13c0aab8,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/common/object-pool.h:34
> #21 0x0000000002a6d18d in impala::QueryState::~QueryState (this=0x13c0a000,
> __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/runtime/query-state.cc:151
> #22 0x0000000002a621e4 in impala::QueryExecMgr::ReleaseQueryState
> (this=0x139e3680, qs=0x0) at
> /home/user/Impala/be/src/runtime/query-exec-mgr.cc:220
> #23 0x0000000004571a0b in impala::Coordinator::~Coordinator (this=0x16307700,
> __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/runtime/coordinator.cc:147
> #24 0x0000000002fc0f63 in boost::checked_delete<impala::Coordinator>
> (x=0x16307700) at
> /opt/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/core/checked_delete.hpp:36
> #25 0x0000000002fbe791 in boost::scoped_ptr<impala::Coordinator>::~scoped_ptr
> (this=0x15e6d588, __in_chrg=<optimized out>) at
> /opt/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/smart_ptr/scoped_ptr.hpp:88
> #26 0x0000000002f9f1b6 in impala::ClientRequestState::~ClientRequestState
> (this=0x15e6cc00, __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/service/client-request-state.cc:195{noformat}
>
> More precisely, it runs into a DCHECK at
> {noformat}
> #8 0x00000000041caa55 in impala::Expr::~Expr (this=0x148bfc00,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/exprs/expr.cc:47 47
> DCHECK(cache_entry_ == nullptr);
> {noformat}
> To reproduce the issue, simply use any native UDF or use the one defined in
> the test-warehouse:
> {noformat}
> create function if not exists default.identity(int) returns int location
> '/test-warehouse/libTestUdfs.so' symbol='Identity';{noformat}
> Interestingly, it still returns the result set as if the update was
> successful and returns the prompt:
> {noformat}
> [localhost:21050] default> UPDATE ice_t SET int_col = identity(int_col);
> Query: UPDATE ice_t SET int_col = identity(int_col)
> Query submitted at: [...]
> Query state can be monitored at: [...]
> Modified 1 row(s) in 0.82s
> [localhost:21050] default> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
