Hello, I'm using Riak in a fairly high-volume scenario and one of my nodes has crashed... I finally found a half-decent reference to innodb crash recovery and what the force_recovery levels mean (here, if anyone's interested: http://www.trickykid.org/article/mysql-forcing-recovery ... dev.mysql.com also has a decent explanation, although it's in the context of using the force_recovery option to allow re-starting a mysql server vs. fixing the actual innodb tables/files)...
I've tried to run innodump (from /priv in the innostore src/build dir) to dump the data with no luck - I get the same error message (first one, below) as when I just try to re-start the Riak node, unless I use -force_recovery 6 (all other levels produce original error), which produces the second error below. Any ideas how I might go about fixing this, or should I assume that node/data is dead/lost and just re-initialize the innostore for this node from scratch? Thanks! (errors follow) --- first error: InnoDB: Mutexes and rw_locks use GCC atomic builtins 101119 21:24:14 InnoDB: highest supported file format is Barracuda. InnoDB: Log scan progressed past the checkpoint lsn 3119545955 101119 21:24:14 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Error: tried to read 16384 bytes at offset 0 14680064. InnoDB: Was only able to read 12288. InnoDB: Fatal error: cannot read from file. OS error number 17. 101119 21:24:14 InnoDB: Assertion failure in thread 3030391696 in file os/os0file.c line 2520 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report, check the InnoDB website for details InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the server startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: the InnoDB website for details InnoDB: about forcing recovery. --- second error: InnoDB: Mutexes and rw_locks use GCC atomic builtins 101119 21:24:46 InnoDB: highest supported file format is Barracuda. InnoDB: The user has set IB_RECOVERY_NO_LOG_REDO on InnoDB: Skipping log redo 101119 21:24:46 Embedded InnoDB 1.0.6.6750 started; log sequence number 0 InnoDB: !!! force_recovery is set to 6 !!! 101119 21:24:46 InnoDB: error: space object of table'innokeystore/click-conversions_0', InnoDB: space id 50 did not exist in memory. Retrying an open. 101119 21:24:46 InnoDB: Assertion failure in thread 3035970448 in file fil/fil0fil.c line 3057 InnoDB: Failing assertion: flags != DICT_TF_COMPACT InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report, check the InnoDB website for details InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the server startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: the InnoDB website for details InnoDB: about forcing recovery. -- Rudy X. Desjardins pgp: 96FFA628 _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
