How much memory does your system have? How much memory is system utilizing before starting Cassandra (use command free)? What are the heap setting it tries to use?
Chris On Sep 15, 2014, at 8:16 PM, Yatong Zhang <bluefl...@gmail.com> wrote: > It's during the startup. I tried to upgrade cassandra from 2.0.7 to 2.0.10, > but looks like cassandra could not start again. Also I found the following > log at '/var/log/messages': > > Sep 16 09:06:59 storage6 kernel: INFO: task java:4971 blocked for more than > 120 seconds. > Sep 16 09:06:59 storage6 kernel: Tainted: G --------------- H > 2.6.32-431.el6.x86_64 #1 > Sep 16 09:06:59 storage6 kernel: "echo 0 > > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > Sep 16 09:06:59 storage6 kernel: java D 0000000000000003 0 4971 > 1 0x00000080 > Sep 16 09:06:59 storage6 kernel: ffff88042b591c98 0000000000000082 > ffffffff81ed4ff0 ffff8803b4f01540 > Sep 16 09:06:59 storage6 kernel: ffff88042b591c68 ffffffff810af370 > ffff88042b591ca0 ffff8803b4f01540 > Sep 16 09:06:59 storage6 kernel: ffff8803b4f01af8 ffff88042b591fd8 > 000000000000fbc8 ffff8803b4f01af8 > Sep 16 09:06:59 storage6 kernel: Call Trace: > Sep 16 09:06:59 storage6 kernel: [<ffffffff810af370>] ? > exit_robust_list+0x90/0x160 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81076ad5>] exit_mm+0x95/0x180 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81076f1f>] do_exit+0x15f/0x870 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81077688>] do_group_exit+0x58/0xd0 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8108d046>] > get_signal_to_deliver+0x1f6/0x460 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100a265>] do_signal+0x75/0x800 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81066629>] ? > wake_up_new_task+0xd9/0x130 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81070ead>] ? do_fork+0x13d/0x480 > Sep 16 09:06:59 storage6 kernel: [<ffffffff810b1c0b>] ? sys_futex+0x7b/0x170 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100aa80>] > do_notify_resume+0x90/0xc0 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100b341>] int_signal+0x12/0x17 > Sep 16 09:06:59 storage6 kernel: INFO: task java:4972 blocked for more than > 120 seconds. > Sep 16 09:06:59 storage6 kernel: Tainted: G --------------- H > 2.6.32-431.el6.x86_64 #1 > Sep 16 09:06:59 storage6 kernel: "echo 0 > > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > Sep 16 09:06:59 storage6 kernel: java D 0000000000000000 0 4972 > 1 0x00000080 > Sep 16 09:06:59 storage6 kernel: ffff8803b4d7fc98 0000000000000082 > ffffffff81ed6d78 ffff8803b4cf1500 > Sep 16 09:06:59 storage6 kernel: ffff8803b4d7fc68 ffffffff810af370 > ffff8803b4d7fca0 ffff8803b4cf1500 > Sep 16 09:06:59 storage6 kernel: ffff8803b4cf1ab8 ffff8803b4d7ffd8 > 000000000000fbc8 ffff8803b4cf1ab8 > Sep 16 09:06:59 storage6 kernel: Call Trace: > Sep 16 09:06:59 storage6 kernel: [<ffffffff810af370>] ? > exit_robust_list+0x90/0x160 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81076ad5>] exit_mm+0x95/0x180 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81076f1f>] do_exit+0x15f/0x870 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81065e20>] ? > wake_up_state+0x10/0x20 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81077688>] do_group_exit+0x58/0xd0 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8108d046>] > get_signal_to_deliver+0x1f6/0x460 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100a265>] do_signal+0x75/0x800 > Sep 16 09:06:59 storage6 kernel: [<ffffffff810097cc>] ? > __switch_to+0x1ac/0x320 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81527910>] ? > thread_return+0x4e/0x76e > Sep 16 09:06:59 storage6 kernel: [<ffffffff810b1c0b>] ? sys_futex+0x7b/0x170 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100aa80>] > do_notify_resume+0x90/0xc0 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100b341>] int_signal+0x12/0x17 > Sep 16 09:06:59 storage6 kernel: INFO: task java:4973 blocked for more than > 120 seconds. > > > On Tue, Sep 16, 2014 at 9:00 AM, Robert Coli <rc...@eventbrite.com> wrote: > On Mon, Sep 15, 2014 at 5:55 PM, Yatong Zhang <bluefl...@gmail.com> wrote: > I just encountered an error which left a log '/hs_err_pid3013.log'. So is > there a way to solve this? > > # There is insufficient memory for the Java Runtime Environment to continue. > # Native memory allocation (malloc) failed to allocate 12288 bytes for > committing reserved memory. > > Use less heap memory? > > You haven't specified under which circumstances this occurred, so I can only > conjecture that it is likely being caused by "writing too fast". > > Write more slowly. > > =Rob > >
