Hello.
We have a storage daemon in a cluster environment, all daemons are the same version ( 9.4.2 ) Virtual tapes are stored in a nfs share, same options and permissions for both nodes. When daemon is running on node 1 all is working well, when daemon is on the other node, we get an "invalid catalog request" error when starting to write to tape (after having correctly identify the tape to use ): bckserver-dir: catreq.c:140-51326 catreq CatReq JobId=51326 GetVolInfo VolName=VolLinux-0379 write=1 bckserver-dir: catreq.c:189-51326 CatReq GetVolInfo Vol=VolLinux-0379 bckserver-dir: next_vol.c:322-51326 Vol=VolLinux-0379 expired=0 bckserver-dir: catreq.c:112-51326 Vol Info for Backup_otbatsp01.2019-08-06_12.06.20_18: 1000 OK VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=56 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 EndFile=3 EndBlock=151747586 VolType=1 LabelType=0 MediaId=379 ScratchPoolId=0 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 Recycle=1 bckserver-dir: catreq.c:430-51326 >CatReq response: 1000 OK VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=56 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 EndFile=3 EndBlock=151747586 VolType=1 LabelType=0 MediaId=379 ScratchPoolId=0 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 Recycle=1 bckserver-dir: catreq.c:431-51326 Leave catreq jcr 0x57f628 bckserver-dir: getmsg.c:151-51326 bget_dirmsg n=60 msglen=60 is_stop=0: CatReq JobId=51326 GetVolInfo VolName=VolLinux-0379 write=1 bckserver-dir: catreq.c:140-51326 catreq CatReq JobId=51326 GetVolInfo VolName=VolLinux-0379 write=1 bckserver-dir: catreq.c:189-51326 CatReq GetVolInfo Vol=VolLinux-0379 bckserver-dir: next_vol.c:322-51326 Vol=VolLinux-0379 expired=0 bckserver-dir: catreq.c:112-51326 Vol Info for Backup_otbatsp01.2019-08-06_12.06.20_18: 1000 OK VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=56 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 EndFile=3 EndBlock=151747586 VolType=1 LabelType=0 MediaId=379 ScratchPoolId=0 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 Recycle=1 bckserver-dir: catreq.c:430-51326 >CatReq response: 1000 OK VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=56 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 EndFile=3 EndBlock=151747586 VolType=1 LabelType=0 MediaId=379 ScratchPoolId=0 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 Recycle=1 bckserver-dir: catreq.c:431-51326 Leave catreq jcr 0x57f628 bckserver-dir: getmsg.c:151-51326 bget_dirmsg n=132 msglen=132 is_stop=0: Jmsg JobId=51326 type=6 level=1565085983 baculafe-sd JobId 51326: Volume "VolLinux-0379" previously written, moving to end of data. bckserver-dir: getmsg.c:151-51326 bget_dirmsg n=402 msglen=402 is_stop=0: CatReq JobId=51326 UpdateMedia VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=57 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 EndTime=1565085983 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 VolFirstWritten=0 VolType=1 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 bckserver-dir: catreq.c:140-51326 catreq CatReq JobId=51326 UpdateMedia VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=57 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 EndTime=1565085983 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 VolFirstWritten=0 VolType=1 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 bckserver-dir: catreq.c:426-51326 Invalid Catalog request: CatReq JobId=51326 UpdateMedia VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=57 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 EndTime=1565085983 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 VolFirstWritten=0 VolType=1 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 bckserver-dir: catreq.c:182-51329 Tried find_media. fields wanted=4, got=-1 bckserver-dir: catreq.c:240-51329 Tried get_vol_info. fields wanted=3, got=-1 bckserver-dir: catreq.c:367-51329 Tried update_media. fields wanted=25, got=-1 bckserver-dir: catreq.c:420-51329 Tried create_jobmedia. fields wanted=10, got=-1 bckserver-dir: catreq.c:430-51326 >CatReq response: 1990 Invalid Catalog Request: CatReq JobId=51326 UpdateMedia VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=57 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 EndTime=1565085983 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 VolFirstWritten=0 VolType=1 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 bckserver-dir: catreq.c:431-51326 Leave catreq jcr 0x57f628 bckserver-dir: getmsg.c:151-51326 bget_dirmsg n=538 msglen=538 is_stop=0: Jmsg JobId=51326 type=3 level=1565085983 baculafe-sd JobId 51326: Fatal error: Error getting Volume info: 1990 Invalid Catalog Request: CatReq JobId=51326 UpdateMedia VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=57 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 EndTime=1565085983 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 VolFirstWritten=0 VolType=1 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 bckserver-dir: getmsg.c:151-51326 bget_dirmsg n=464 msglen=464 is_stop=0: Jmsg Job=Backup_otbatsp01.2019-08-06_12.06.20_18 type=3 level=1565085983 otbatsp01-fd JobId 51326: Fatal error: job.c:2484 Bad response from SD to Append Data command. Wanted 3000 OK data , got len=484 msg="3903 Error append data: Error getting Volume info: 1990 Invalid Catalog Request: CatReq JobId=51326 UpdateMedia VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=57 VolErrors=0" When running with no error, in trace we see: bckserver-dir: getmsg.c:253-51330 Catalog req jcr=10057f628: CatReq JobId=51330 UpdateMedia VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=57 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 EndTime=1565094536 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 VolFirstWritten=0 VolType=1 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 Recycle=1 bckserver-dir: catreq.c:140-51330 catreq CatReq JobId=51330 UpdateMedia VolName=VolLinux-0379 VolJobs=2 VolFiles=3 VolBlocks=202083 VolBytes=13036649475 VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=57 VolErrors=0 VolWrites=6027553 MaxVolBytes=53687091200 EndTime=1565094536 VolStatus=Append Slot=0 relabel=0 InChanger=0 VolReadTime=0 VolWriteTime=16050170606 VolFirstWritten=0 VolType=1 VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1 Recycle=1 bckserver-dir: catreq.c:182-51330 Tried find_media. fields wanted=4, got=-1 bckserver-dir: catreq.c:240-51330 Tried get_vol_info. fields wanted=3, got=-1 bckserver-dir: catreq.c:259-51330 Update media VolLinux-0379 oldStat= newStat=Append bckserver-dir: catreq.c:298-51330 Update media: BefVolJobs=2 After=2 bckserver-dir: catreq.c:350-51330 db_update_media_record. Stat=Append Vol=VolLinux-0379 I'm pretty sure that in the past storage daemon was working well on both nodes. IMPORTANT NOTICE This message may contain privileged or confidential information. If you are not the intended recipient please notify the sender and delete the message from your computer immediately. Any dissemination, distribution or copy is strictly forbidden. Please note that e-mails may be faked, contain viruses or not assure receipt, without any responsibility for our company.
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
