It's during the startup. I tried to upgrade cassandra from 2.0.7 to 2.0.10, but looks like cassandra could not start again. Also I found the following log at '/var/log/messages':
Sep 16 09:06:59 storage6 kernel: INFO: task java:4971 blocked for more than > 120 seconds. > Sep 16 09:06:59 storage6 kernel: Tainted: G --------------- > H 2.6.32-431.el6.x86_64 #1 > Sep 16 09:06:59 storage6 kernel: "echo 0 > > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > Sep 16 09:06:59 storage6 kernel: java D 0000000000000003 0 > 4971 1 0x00000080 > Sep 16 09:06:59 storage6 kernel: ffff88042b591c98 0000000000000082 > ffffffff81ed4ff0 ffff8803b4f01540 > Sep 16 09:06:59 storage6 kernel: ffff88042b591c68 ffffffff810af370 > ffff88042b591ca0 ffff8803b4f01540 > Sep 16 09:06:59 storage6 kernel: ffff8803b4f01af8 ffff88042b591fd8 > 000000000000fbc8 ffff8803b4f01af8 > Sep 16 09:06:59 storage6 kernel: Call Trace: > Sep 16 09:06:59 storage6 kernel: [<ffffffff810af370>] ? > exit_robust_list+0x90/0x160 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81076ad5>] exit_mm+0x95/0x180 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81076f1f>] do_exit+0x15f/0x870 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81077688>] > do_group_exit+0x58/0xd0 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8108d046>] > get_signal_to_deliver+0x1f6/0x460 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100a265>] do_signal+0x75/0x800 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81066629>] ? > wake_up_new_task+0xd9/0x130 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81070ead>] ? do_fork+0x13d/0x480 > Sep 16 09:06:59 storage6 kernel: [<ffffffff810b1c0b>] ? > sys_futex+0x7b/0x170 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100aa80>] > do_notify_resume+0x90/0xc0 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100b341>] int_signal+0x12/0x17 > Sep 16 09:06:59 storage6 kernel: INFO: task java:4972 blocked for more > than 120 seconds. > Sep 16 09:06:59 storage6 kernel: Tainted: G --------------- > H 2.6.32-431.el6.x86_64 #1 > Sep 16 09:06:59 storage6 kernel: "echo 0 > > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > Sep 16 09:06:59 storage6 kernel: java D 0000000000000000 0 > 4972 1 0x00000080 > Sep 16 09:06:59 storage6 kernel: ffff8803b4d7fc98 0000000000000082 > ffffffff81ed6d78 ffff8803b4cf1500 > Sep 16 09:06:59 storage6 kernel: ffff8803b4d7fc68 ffffffff810af370 > ffff8803b4d7fca0 ffff8803b4cf1500 > Sep 16 09:06:59 storage6 kernel: ffff8803b4cf1ab8 ffff8803b4d7ffd8 > 000000000000fbc8 ffff8803b4cf1ab8 > Sep 16 09:06:59 storage6 kernel: Call Trace: > Sep 16 09:06:59 storage6 kernel: [<ffffffff810af370>] ? > exit_robust_list+0x90/0x160 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81076ad5>] exit_mm+0x95/0x180 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81076f1f>] do_exit+0x15f/0x870 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81065e20>] ? > wake_up_state+0x10/0x20 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81077688>] > do_group_exit+0x58/0xd0 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8108d046>] > get_signal_to_deliver+0x1f6/0x460 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100a265>] do_signal+0x75/0x800 > Sep 16 09:06:59 storage6 kernel: [<ffffffff810097cc>] ? > __switch_to+0x1ac/0x320 > Sep 16 09:06:59 storage6 kernel: [<ffffffff81527910>] ? > thread_return+0x4e/0x76e > Sep 16 09:06:59 storage6 kernel: [<ffffffff810b1c0b>] ? > sys_futex+0x7b/0x170 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100aa80>] > do_notify_resume+0x90/0xc0 > Sep 16 09:06:59 storage6 kernel: [<ffffffff8100b341>] int_signal+0x12/0x17 > Sep 16 09:06:59 storage6 kernel: INFO: task java:4973 blocked for more > than 120 seconds. > On Tue, Sep 16, 2014 at 9:00 AM, Robert Coli <rc...@eventbrite.com> wrote: > On Mon, Sep 15, 2014 at 5:55 PM, Yatong Zhang <bluefl...@gmail.com> wrote: > >> I just encountered an error which left a log '/hs_err_pid3013.log'. So is >> there a way to solve this? >> >> # There is insufficient memory for the Java Runtime Environment to >>> continue. >>> # Native memory allocation (malloc) failed to allocate 12288 bytes for >>> committing reserved memory. >>> >> > Use less heap memory? > > You haven't specified under which circumstances this occurred, so I can > only conjecture that it is likely being caused by "writing too fast". > > Write more slowly. > > =Rob > >
