Thread starting here: https://www.postgresql.org/message-id/20201001021609.GC8476%40telsasoft.com
On Fri, Dec 18, 2020 at 05:56:07PM -0600, Justin Pryzby wrote: > I'm 99% sure the "bad_alloc" is from LLVM. It happened multiple times on > different servers (running a similar report) after setting jit=on during pg13 > upgrade, and never happened since re-setting jit=off. Since this recurred a few times recently (now running pg14.0), and I finally managed to get a non-truncated corefile... Core was generated by `postgres: telsasoft ts 192.168.122.13(40808) SELECT '. Program terminated with signal 6, Aborted. #0 0x00007f2497880337 in raise () from /lib64/libc.so.6 #1 0x00007f2497881a28 in abort () from /lib64/libc.so.6 #2 0x00007f2487cbf265 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib64/llvm5.0/lib/libLLVM-5.0.so #3 0x00007f2487c66696 in __cxxabiv1::__terminate(void (*)()) () from /usr/lib64/llvm5.0/lib/libLLVM-5.0.so #4 0x00007f2487c666c3 in std::terminate() () from /usr/lib64/llvm5.0/lib/libLLVM-5.0.so #5 0x00007f2487c687d3 in __cxa_throw () from /usr/lib64/llvm5.0/lib/libLLVM-5.0.so #6 0x00007f2487c686cd in operator new(unsigned long) () from /usr/lib64/llvm5.0/lib/libLLVM-5.0.so #7 0x00007f2486477b9c in llvm::DenseMap<llvm::APInt, std::unique_ptr<llvm::ConstantInt, std::default_delete<llvm::ConstantInt> >, llvm::DenseMapAPIntKeyInfo, llvm::detail::DenseMapPair<llvm::APInt, std::unique_ptr<llvm::ConstantInt, std::default_delete<llvm::ConstantInt> > > >::grow(unsigned int) () from /usr/lib64/llvm5.0/lib/libLLVM-5.0.so #8 0x00007f2486477f29 in llvm::ConstantInt::get(llvm::LLVMContext&, llvm::APInt const&) () from /usr/lib64/llvm5.0/lib/libLLVM-5.0.so #9 0x00007f2486478263 in llvm::ConstantInt::get(llvm::IntegerType*, unsigned long, bool) () from /usr/lib64/llvm5.0/lib/libLLVM-5.0.so #10 0x00007f2488f66c18 in l_ptr_const (type=0x3000650, ptr=<optimized out>) at ../../../../src/include/jit/llvmjit_emit.h:29 #11 llvm_compile_expr (state=<optimized out>) at llvmjit_expr.c:246 #12 0x0000000000635499 in ExecReadyExpr (state=state@entry=0x1a5317178) at execExpr.c:885 #13 0x00000000006388a5 in ExecInitQual (qual=0x1a6d07de8, parent=parent@entry=0x1a4c4ade0) at execExpr.c:280 #14 0x0000000000661d9d in ExecInitHashJoin (node=node@entry=0x1a6c935b0, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeHashjoin.c:733 #15 0x0000000000648f6e in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:307 #16 0x000000000065e8fd in ExecInitHash (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at nodeHash.c:381 #17 0x0000000000648e87 in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:365 #18 0x0000000000661cd5 in ExecInitHashJoin (node=node@entry=0x1a6c93ce0, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeHashjoin.c:663 #19 0x0000000000648f6e in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:307 #20 0x00000000006723e4 in ExecInitSort (node=node@entry=0x1a6c94a50, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeSort.c:210 #21 0x0000000000648f2f in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:320 #22 0x0000000000655ef9 in ExecInitAgg (node=node@entry=0x1a6c95100, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeAgg.c:3382 #23 0x0000000000648ef0 in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:340 #24 0x0000000000674628 in ExecInitSubqueryScan (node=node@entry=0x1a6c94020, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeSubqueryscan.c:126 #25 0x0000000000649055 in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:249 #26 0x0000000000658e61 in ExecInitAppend (node=node@entry=0x1a6c60008, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeAppend.c:232 #27 0x0000000000649166 in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:181 #28 0x0000000000670e1a in ExecInitResult (node=node@entry=0x1a6c95c40, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeResult.c:210 #29 0x00000000006491a5 in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:166 #30 0x0000000000658e61 in ExecInitAppend (node=node@entry=0x1a6c5d548, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeAppend.c:232 #31 0x0000000000649166 in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:181 #32 0x0000000000674628 in ExecInitSubqueryScan (node=node@entry=0x1a6ca5368, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeSubqueryscan.c:126 #33 0x0000000000649055 in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:249 #34 0x00000000006723e4 in ExecInitSort (node=node@entry=0x1a6ca5a28, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeSort.c:210 #35 0x0000000000648f2f in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:320 #36 0x00000000006793eb in ExecInitWindowAgg (node=node@entry=0x1a6ca6ab0, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeWindowAgg.c:2314 #37 0x0000000000648edb in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:345 #38 0x0000000000674628 in ExecInitSubqueryScan (node=node@entry=0x1a6ca5628, estate=estate@entry=0x1a4da87b8, eflags=eflags@entry=16) at nodeSubqueryscan.c:126 #39 0x0000000000649055 in ExecInitNode (node=<optimized out>, estate=<optimized out>, eflags=<optimized out>) at execProcnode.c:249 #40 0x0000000000642ab1 in InitPlan (eflags=16, queryDesc=0x344dc88) at execMain.c:936 #41 standard_ExecutorStart (queryDesc=0x344dc88, eflags=16) at execMain.c:263 #42 0x00007f248b45656d in pgss_ExecutorStart (queryDesc=0x344dc88, eflags=<optimized out>) at pg_stat_statements.c:965 #43 0x00007f248b03a861 in explain_ExecutorStart (queryDesc=0x344dc88, eflags=<optimized out>) at auto_explain.c:301 #44 0x00000000007b8d1b in PortalStart (portal=0x1be2168, params=0x0, eflags=0, snapshot=0x0) at pquery.c:512 -- Justin
