Today, I was reminded why to be careful with SHOW commands during
production workloads and thought I'd share (and archive across the
Internet at large) by posting here:
03/04/06 10:44:46 ANR4391I Expiration processing node NODE, filespace
/oracle, fsId 141, domain STANDARD, and management class DEFAULT -
for BACKUP type files. (SESSION: 64389, PROCESS: 907)
03/04/06 10:44:50 ANR2017I Administrator ADMIN issued command: show
dbreorg (SESSION: 71539)
03/04/06 10:44:50 ANR9999D pkthread.c(593): ThreadId<66> Run-time
assertion failed:
"tmCmpTsn( tmGetTsn( txnP->tid ), tmGetTsn( tid ) ) == EQUAL",
Thread 66, File dbtxn.c, Line 331. (SESSION: 71539)
03/04/06 10:44:50 ANR9999D ThreadId<66> issued message 9999 from:
<-0x000000010001c168 outDiagf
<-0x00000001000103f0 pkLogicAbort
<-0x000000010029cef0 dbParticipate
<-0x00000001000b85c8 tbOpen
<-0x000000010003ce44 admAuthSystem
<-0x00000001007051c8 AdmEstimateDbReorg
<-0x00000001007a69f0 AdmShow
<-0x00000001001d92d4 AdmCommandLocal
<-0x00000001001da680 admCommand
<-0x000000010052c294 SmAdminCommandThread
<-0x000000010000e9dc StartThread
<-0x090000000036c50c _pthread_body (SESSION: 71539)
03/04/2006 10:44:50 ANR7838S Server operation terminated.
03/04/2006 10:44:50 ANR7837S Internal error DBTXN6773 detected.
03/04/2006 10:44:50 ANR7833S Server thread 1 terminated in response to
program abort.
03/04/2006 10:44:50 ANR7833S Server thread 2 terminated in response to
program abort.
03/04/2006 10:44:50 ANR7833S Server thread 3 terminated in response to
program abort.
...
Note that immediately preceding this, here were the processes:
tsm: EARNHARDT>q proc
Process Process Description Status
Number
------- -------------------- -------------------------------------------------
907 Expiration Examined 13472440 objects, deleting 316425 backup
objects, 1659 archive objects, 0 DB backup
volumes, 0 recovery plan files; 0 errors
encountered.
917 Migration Disk Storage Pool DISKCOL, Moved Files: 855,
Moved Bytes: 230,914,351,104, Unreadable Files:
0, Unreadable Bytes: 0. Current Physical File
(bytes): 1,369,759,744 Current output volume:
M00758.
I think this might be related to the defect posted by Orville Lantto a few
days ago:
IC48429: MIGRATE STGPOOL PREVENTS UPDATE STGPOOL FROM RUNNING AND HANGS
SUBSEQUENT QUERY VOLUME AND SELECT FROM VOLUMES COMMANDS.
I've found that the hangs are more than just from UPD STG, but also from
other lock intensive processes.
According to Orville, avoiding the new commands for MIGRATE STG and
RECLAIM STG makes the problems go away.
But, since the crash occurred during a SHOW command, which is supposed to
be a developer help/backdoor, there is/would be no support for this crash.
-Josh
