Hi Asai, Secondary storage garbage collector (runs every "storage.cleanup.interval" secs) tries to cleanup all the destroyed volume snapshots. The log below has no relation with VM snapshots. As per the log, the cleanup failed for a volume snapshot as Cloudstack couldn't get the details in snapshot_store_ref table. Any out of band db operations performed?
Can you please send the complete cleanup failure log and the db records of snapshot_store_ref table for destroyed / cleanup failed snapshots? Have you observed any errors related to VM snapshots in the log? - Suresh On Fri, May 3, 2019 at 5:09 AM Asai <[email protected]> wrote: > Greetings, > > I have been doing some maintenance on my Cloudstack instance (4.11.1.0) > and I have found that some of my VM snapshots quit running back in > February. I have found this error in the management-server.log. > > 2019-05-02 10:01:21,547 DEBUG [c.c.s.StorageManagerImpl] > (StorageManager-Scavenger-1:ctx-5b4e7b0e) (logid:00439c5c) Secondary > storage garbage collector found 252 snapshots to cleanup on > snapshot_store_ref for store: Avalon Secondary Storage > 2019-05-02 10:01:21,552 WARN [c.c.s.StorageManagerImpl] > (StorageManager-Scavenger-1:ctx-5b4e7b0e) (logid:00439c5c) problem cleaning > up snapshots in snapshot_store_ref for store: Avalon Secondary Storage > at > org.apache.cloudstack.storage.snapshot.SnapshotObject.getId(SnapshotObject.java:159) > at > org.apache.cloudstack.storage.snapshot.SnapshotObject.getChild(SnapshotObject.java:124) > > Can anyone shed some light on this, and advise how I can get my snapshots > running again? > Asai > > >
