Hello, I've recently experienced three segfaults from ods-signer. In all three cases the log contains errors about keys that can not be found. I initially assumed they were erroneously deleted from the HSM but 'ods-hsmutil' is able to find them. After restarting the signer it seems to work fine.
I'm using OpenDNSSEC 1.3.9-5 and SoftHSM 1.3.3-2 as provided by Debian/wheezy. The logs below have been abbreviated and redacted. I'll provide full logs upon request. I've increased logging and enabled coredumps, so if it happens again I may have more information. Crash on host Ramanujan Mar 18 00:10:53 ramanujan ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 00:10:53 ramanujan ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 00:10:53 ramanujan ods-enforcerd: Key remove successful. Mar 18 00:10:53 ramanujan ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 00:10:53 ramanujan ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 00:10:53 ramanujan ods-enforcerd: Key remove successful. Mar 18 00:10:54 ramanujan ods-signerd: [hsm] idle libhsm connection, trying to reopen Mar 18 00:10:55 ramanujan ods-signerd: [hsm] hsm_get_slot_id(): could not find token with the name LocalHSM Mar 18 00:10:55 ramanujan ods-signerd: [hsm] unable to get key: key d1f03a7b14eac19b355e23ce1b47d0d9 not found Mar 18 00:10:55 ramanujan ods-signerd: [hsm] unable to sign: get key failed Mar 18 00:10:55 ramanujan ods-signerd: [rrset] unable to sign RRset[6]: error creating RRSIG RR Mar 18 00:10:55 ramanujan ods-signerd: [worker[4]] sign zone example3.nl failed: 1 of 11 signatures failed Mar 18 00:10:55 ramanujan ods-signerd: [worker[4]] backoff task [sign] for zone example3.nl with 60 seconds ... same for many other zones ... Mar 18 00:14:02 ramanujan ods-signerd: [hsm] unable to get key: key d56c511a78d2a9406b0d135edc80a758 not found Mar 18 00:14:02 ramanujan ods-signerd: [hsm] unable to get key: key d56c511a78d2a9406b0d135edc80a758 not found Mar 18 00:14:02 ramanujan ods-signerd: [hsm] unable to sign: get key failed Mar 18 00:14:02 ramanujan ods-signerd: [rrset] unable to sign RRset[6]: error creating RRSIG RR Mar 18 00:14:02 ramanujan ods-signerd: [worker[7]] sign zone example3.org failed: 1 of 10 signatures failed Mar 18 00:14:02 ramanujan ods-signerd: [worker[7]] backoff task [sign] for zone example3.org with 120 seconds Mar 18 00:14:03 ramanujan ods-signerd: [hsm] unable to get key: key cf06b88e9a3867461af1bd628fda4d51 not found Mar 18 00:14:03 ramanujan ods-signerd: [hsm] unable to get key: key cf06b88e9a3867461af1bd628fda4d51 not found Mar 18 00:14:03 ramanujan ods-signerd: [hsm] unable to sign: get key failed Mar 18 00:14:03 ramanujan ods-signerd: [rrset] unable to sign RRset[6]: error creating RRSIG RR Mar 18 00:14:03 ramanujan ods-signerd: [worker[7]] sign zone example4.org failed: 1 of 13 signatures failed Mar 18 00:14:03 ramanujan ods-signerd: [rrset] unable to sign RRset[6]: error creating RRSIG RR Mar 18 00:14:03 ramanujan ods-signerd: [worker[7]] sign zone example4.org failed: 1 of 13 signatures failed Mar 18 00:14:03 ramanujan ods-signerd: [worker[7]] backoff task [sign] for zone example4.org with 120 seconds Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to get key: key 462167fef14dff802a768a2234003d60 not found Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to get key: key 462167fef14dff802a768a2234003d60 not found Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to sign: get key failed Mar 18 00:14:20 ramanujan ods-signerd: [rrset] unable to sign RRset[6]: error creating RRSIG RR Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to get key: key 462167fef14dff802a768a2234003d60 not found Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to get key: key 462167fef14dff802a768a2234003d60 not found Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to sign: get key failed Mar 18 00:14:20 ramanujan ods-signerd: [rrset] unable to sign RRset[12]: error creating RRSIG RR Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to get key: key 462167fef14dff802a768a2234003d60 not found Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to get key: key 462167fef14dff802a768a2234003d60 not found Mar 18 00:14:20 ramanujan ods-signerd: [hsm] unable to get key: key 462167fef14dff802a768a2234003d60 not found Mar 18 00:14:20 ramanujan kernel: [1059710.358951] ods-signerd[24797]: segfault at 10000000010 ip 00007fbb353b206a sp 00007fbb2ed4d490 error 4 in libc-2.13.so[7fbb3533b000+180000] First crash on host Metagross Mar 14 05:10:36 metagross ods-signerd: [STATS] example.eu RR[count=0 time=0(sec)] NSEC3[count=0 time=0(sec)] RRSIG[new=1 reused=9 time=3(sec) avg=0(sig/sec)] AUDIT[time=0(sec)] TOTAL[time=3(sec)] Mar 14 05:10:36 metagross ods-enforcerd: Purging keys... Mar 14 05:10:36 metagross ods-signerd: [STATS] example2.eu RR[count=0 time=0(sec)] NSEC3[count=0 time=0(sec)] RRSIG[new=1 reused=8 time=2(sec) avg=0(sig/sec)] AUDIT[time=0(sec)] TOTAL[time=2(sec)] Mar 14 05:10:37 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 14 05:10:37 metagross ods-signerd: [hsm] idle libhsm connection, trying to reopen Mar 14 05:10:37 metagross ods-signerd: [hsm] idle libhsm connection, trying to reopen Mar 14 05:10:37 metagross ods-signerd: ../../../signer/src/daemon/engine.c at 367 could not pthread_join(engine->drudgers[i]->thread_id, NULL): Invalid argument Mar 14 05:10:37 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 14 05:10:37 metagross ods-enforcerd: Key remove successful. Mar 14 05:10:38 metagross ods-enforcerd: Policy standbyyourkey found. Mar 14 05:10:38 metagross ods-enforcerd: Key sharing is Off. Mar 14 05:10:38 metagross ods-enforcerd: No zones on policy standbyyourkey, skipping... Mar 14 05:10:38 metagross ods-enforcerd: Purging keys... Mar 14 05:10:38 metagross ods-signerd: ../../../signer/src/daemon/engine.c at 367 could not pthread_join(engine->drudgers[i]->thread_id, NULL): Invalid argument Mar 14 05:10:39 metagross kernel: [52691.560115] ods-signerd[3896]: segfault at 7f30a8a3a9d0 ip 00007f30b83bad8c sp 00007f30b4a41dc0 error 4 in libpthread-2.13.so[7f30b83b3000+17000] Second crash on host Metagross Mar 18 05:10:28 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:28 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:28 metagross ods-enforcerd: Key remove successful. ... many more ... Mar 18 05:10:41 metagross ods-signerd: [hsm] idle libhsm connection, trying to reopen Mar 18 05:10:41 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:41 metagross ods-enforcerd: Key remove successful. Mar 18 05:10:41 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:41 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:41 metagross ods-enforcerd: Key remove successful. Mar 18 05:10:41 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:42 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:42 metagross ods-enforcerd: Key remove successful. Mar 18 05:10:42 metagross ods-signerd: [hsm] idle libhsm connection, trying to reopen Mar 18 05:10:42 metagross kernel: [398294.951400] ods-signerd[28480]: segfault at 7f352f47b9d0 ip 00007f3535df9d8c sp 00007f3531c7fdc0 error 4 in libpthread-2.13.so[7f3535df2000+17000] Mar 18 05:10:42 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:42 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:42 metagross ods-enforcerd: Key remove successful. Mar 18 05:10:42 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:42 metagross ods-enforcerd: SoftHSM: C_DestroyObject: An object has been destroyed Mar 18 05:10:42 metagross ods-enforcerd: Key remove successful. ... enforcer continues destroying ... -- Casper Gielen <cgie...@uvt.nl> | LIS UNIX PGP fingerprint = 16BD 2C9F 8156 C242 F981 63B8 2214 083C F80E 4AF7 Universiteit van Tilburg | Postbus 90153, 5000 LE Warandelaan 2 | Telefoon 013 466 4100 | G 236 | http://www.uvt.nl _______________________________________________ Opendnssec-user mailing list Opendnssec-user@lists.opendnssec.org https://lists.opendnssec.org/mailman/listinfo/opendnssec-user
