Overview Riak KV (aka Riak / Riak EE) 2.1.3 has a race condition with opening LevelDB databases and may crash with a segmentation violation (SIGSEGV). LevelDB databases are opened on primary or fallback vnode startup and during AAE expiration The issue is possible but much less likely on the Riak TS 2.0, Riak KV 2.0 and earlier Riak KV 2.1 series.
Description Beginning with Riak KV 2.1.3, a list of opened databases is used by the GroomingCompactions feature to look for databases with low write activity that can benefit from being compacted. The race condition occurs because newly opened databases are added to the list before their initialization is complete. Scanning for GroomedCompaction candidates takes place once every minute or whenever one completes. If the initializing database is accessed during initialization, it may reference uninitialized memory and crash with a segmentation violation. The open database list was originally added to support the FlexCache feature in Riak 2.0. There is a much smaller chance this race condition will occur in Riak 2.0, since it requires more than one database to open or close at the same time. This issue is corrected by ensuring that the list of open databases is updated after the initialization of the database is complete. The correction is shipped within Riak KV 2.0.7, Riak KV 2.1.4, and Riak TS 1.3.0 Affected Users - Users running Riak KV 2.1.3 with the LevelDB backend, with or without AAE enabled. - Users running Riak KV 2.1.3 with other backends (multi, memory, bitcask) and AAE enabled could also hit this issue. - Users running Riak KV 2.0.0 and higher with AAE enabled are also affected, but the likelihood of hitting the race condition is much lower. - Users running Riak TS 1.0.0 and higher are also affected, but the likelihood of hitting the race condition is much lower. Impact An affected node will reference uninitialized memory and will likely exit with a segmentation violation. It may be logged by the operating system as a crash by the beam.smp process. Mitigation Strategy Upgrade to a release of Riak KV at 2.0.7⁄2.1.4 or later to resolve the issue, or patch the eleveldb.so shared library. Riak KV 2.1.4 is the same as Riak KV 2.1.3 with a corrected version of the eleveldb.so library. Riak KV 2.1.4 is available at our downloads site here: http://docs.basho.com/riak/kv/2.1.4/downloads/ Riak KV 2.0.7 is a regular patch release that is in progress and will be released in the next few weeks. If you are unable to upgrade/patch, the occurrence can be reduced by temporarily disabling AAE until the node is upgraded or patched. Instructions Installation and upgrade instructions can be found here http://docs.basho.com/community/productadvisories/leveldbsegfault/ -- Seema Jethani Director of Product Management, Basho <http://basho.com> 4083455739 | @seemaj <http://twitter.com/seemaj>
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
