Hi,
Just had a go at upgrading a node to the latest stable c* 2 release and
think I ran into some issues with manifest migration.
On initial start up I hit this error as it starts to load the first of my
CF.
INFO [main] 2013-09-24 22:56:01,018 LegacyLeveledManifest.java (line 89)
Migrating manifest for struqrealtime/impressionstorev2
INFO [main] 2013-09-24 22:56:01,019 LegacyLeveledManifest.java (line 119)
Snapshotting struqrealtime, impressionstorev2 to pre-sstablemetamigration
ERROR [main] 2013-09-24 22:56:01,030 CassandraDaemon.java (line 459)
Exception encountered during startup
FSWriteError in
/disk1/cassandra/data/struqrealtime/impressionstorev2/snapshots/pre-sstablem
etamigration/impressionstorev2.json
at
org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:83)
at
org.apache.cassandra.db.compaction.LegacyLeveledManifest.snapshotWithoutCFS(
LegacyLeveledManifest.java:138)
at
org.apache.cassandra.db.compaction.LegacyLeveledManifest.migrateManifests(Le
gacyLeveledManifest.java:91)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:246)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:4
42)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:485)
Caused by: java.nio.file.NoSuchFileException:
/disk1/cassandra/data/struqrealtime/impressionstorev2/snapshots/pre-sstablem
etamigration/impressionstorev2.json ->
/disk1/cassandra/data/struqrealtime/impressionstorev2/impressionstorev2.json
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at
sun.nio.fs.UnixFileSystemProvider.createLink(UnixFileSystemProvider.java:474
)
at java.nio.file.Files.createLink(Files.java:1037)
at
org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:79)
... 5 more
I had already successful run a test migration on our dev server. Only real
difference I can see if the number of data directories defined and the
amount of data being held.
I've run upgradesstables under 1.2.10. I have always been using vnodes and
CQL3. I recently moved to using LZ4 instead of Snappy..
I tried to startup again and it gave me a slightly different error
INFO [main] 2013-09-24 22:58:28,218 LegacyLeveledManifest.java (line 89)
Migrating manifest for struqrealtime/impressionstorev2
INFO [main] 2013-09-24 22:58:28,218 LegacyLeveledManifest.java (line 119)
Snapshotting struqrealtime, impressionstorev2 to pre-sstablemetamigration
ERROR [main] 2013-09-24 22:58:28,222 CassandraDaemon.java (line 459)
Exception encountered during startup
java.lang.RuntimeException: Tried to create duplicate hard link to
/disk3/cassandra/data/struqrealtime/impressionstorev2/snapshots/pre-sstablem
etamigration/struqrealtime-impressionstorev2-ic-1030-TOC.txt
at
org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:71)
at
org.apache.cassandra.db.compaction.LegacyLeveledManifest.snapshotWithoutCFS(
LegacyLeveledManifest.java:129)
at
org.apache.cassandra.db.compaction.LegacyLeveledManifest.migrateManifests(Le
gacyLeveledManifest.java:91)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:246)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:4
42)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:485)
Will have a go recreating this tomorrow.
Any insight or guesses at what the issue might be are always welcome.
Thanks,
Chris
