Hi Silver
I've made a test using your script and worked fine!
I'm using bacula 7.4.0.
- MySQL user has all privileges;
grant all on *.* to 'user'@'localhost' identified by 'password';
- The directory and file /mnt/fifo/intranet.sql doesn't exists
- Permission of script is 775
##################################################
# FileSet teste bpipe
FileSet {
Name = "FileSet_Bpipe_Bacula"
Include {
Options {
signature = md5
compression = gzip
}
Plugin = "bpipe:/backup/dump_mysql.sql:/usr/local/bin/backup_mysql.sh
intranet:/usr/local/bin/backup_mysql.sh -R"
}
}
##################################################
Backup Log
2016-01-22 13:43:01 bacula-dir JobId 3012: Start Backup JobId 3012,
Job=Backup_Bpipe_Bacula.2016-01-22_13.42.59_48
2016-01-22 13:43:01 bacula-dir JobId 3012: Using Device
"DiscoLocal1" to write.
2016-01-22 13:43:01 bacula-sd JobId 3012: Volume
"Volume-Default-0001" previously written, moving to end of data.
2016-01-22 13:43:01 bacula-sd JobId 3012: Ready to append to end of
Volume "Volume-Default-0001" size=35,757,662
2016-01-22 13:43:05 bacula-sd JobId 3012: Elapsed time=00:00:04,
Transfer rate=1.680 M Bytes/second
2016-01-22 13:43:05 bacula-sd JobId 3012: Sending spooled attrs to
the Director. Despooling 285 bytes ...
2016-01-22 13:43:05 bacula-dir JobId 3012: Bacula bacula-dir 7.4.0 (16Jan16):
Build OS: x86_64-unknown-linux-gnu debian 8.2
JobId: 3012
Job: Backup_Bpipe_Bacula.2016-01-22_13.42.59_48
Backup Level: Full
Client: "srv_bacula-fd" 7.4.0 (16Jan16)
x86_64-unknown-linux-gnu,debian,8.2
FileSet: "FileSet_Bpipe_Bacula" 2016-01-22 13:42:59
Pool: "Pool-Default" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "StorageLocal1" (From Pool resource)
Scheduled time: 22-Jan-2016 13:42:59
Start time: 22-Jan-2016 13:43:01
End time: 22-Jan-2016 13:43:05
Elapsed time: 4 secs
Priority: 10
FD Files Written: 1
SD Files Written: 1
FD Bytes Written: 6,722,331 (6.722 MB)
SD Bytes Written: 6,722,542 (6.722 MB)
Rate: 1680.6 KB/s
Software Compression: 85.1% 6.7:1
Snapshot/VSS: no
Encryption: no
Accurate: no
Volume name(s): Volume-Default-0001
Volume Session Id: 7
Volume Session Time: 1453470218
Last Volume Bytes: 42,492,736 (42.49 MB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
2016-01-22 13:43:05 bacula-dir JobId 3012: Begin pruning Jobs older
than 1 year .
2016-01-22 13:43:05 bacula-dir JobId 3012: No Jobs found to prune.
2016-01-22 13:43:05 bacula-dir JobId 3012: Begin pruning Files.
2016-01-22 13:43:05 bacula-dir JobId 3012: No Files found to prune.
2016-01-22 13:43:05 bacula-dir JobId 3012: End auto prune.
2016-01-22 13:43:05 bacula-dir JobId 3012: shell command: run
AfterJob "/etc/bacula/scripts/send_whatsapp.sh "Client=srv_bacula-fd"
"JobId=3012" "JobName=Backup_Bpipe_Bacula" "JobBytes=6722331"
"JobFiles=1" "Level=Full" "Pool=Pool-Default" "Storage=StorageLocal1"
"Status=OK""
2016-01-22 13:43:06 bacula-dir JobId 3012: AfterJob:
INFO:yowsup.demos.sendclient.layer:Message sent
Restore Log
2016-01-22 13:44:44 bacula-dir JobId 3013: Start Restore Job
Restore_Files.2016-01-22_13.44.42_56
2016-01-22 13:44:44 bacula-dir JobId 3013: Using Device "DiscoLocal1" to read.
2016-01-22 13:44:44 bacula-sd JobId 3013: Ready to read from volume
"Volume-Default-0001" on file device "DiscoLocal1" (/mnt/disco_usb).
2016-01-22 13:44:44 bacula-sd JobId 3013: Forward spacing Volume
"Volume-Default-0001" to file:block 0:35757662.
2016-01-22 13:44:45 bacula-sd JobId 3013: End of Volume at file 0 on
device "DiscoLocal1" (/mnt/disco_usb), Volume "Volume-Default-0001"
2016-01-22 13:44:45 bacula-sd JobId 3013: End of all volumes.
2016-01-22 13:44:45 bacula-sd JobId 3013: Elapsed time=00:00:01,
Transfer rate=6.722 M Bytes/second
2016-01-22 13:45:57 bacula-dir JobId 3013: Bacula bacula-dir 7.4.0 (16Jan16):
Build OS: x86_64-unknown-linux-gnu debian 8.2
JobId: 3013
Job: Restore_Files.2016-01-22_13.44.42_56
Restore Client: srv_bacula-fd
Start time: 22-Jan-2016 13:44:44
End time: 22-Jan-2016 13:45:57
Files Expected: 1
Files Restored: 1
Bytes Restored: 45,185,460
Rate: 619.0 KB/s
FD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Restore OK
2016-01-22 13:45:57 bacula-dir JobId 3013: Begin pruning Jobs older
than 1 year .
2016-01-22 13:45:57 bacula-dir JobId 3013: No Jobs found to prune.
2016-01-22 13:45:57 bacula-dir JobId 3013: Begin pruning Files.
2016-01-22 13:45:57 bacula-dir JobId 3013: No Files found to prune.
2016-01-22 13:45:57 bacula-dir JobId 3013: End auto prune.
Best Regards
Wanderlei
2016-01-22 11:21 GMT-02:00 Silver Salonen <silver.salo...@gmail.com>:
> Debug output concerning mysqldump from /opt/bacula/bin/bacula-fd -d 200 -c
> bacula-fd.conf -f
>
> job.c:1227-54555 P bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> fd_plugins.c:329-54555 plugin cmd=bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> fd_plugins.c:127-54555 name=bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> len=5 plugin=bpipe-fd.so plen=5
> bpipe-fd.c:215-54555 bpipe-fd:
> PluginCommand=bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> accurate.c:230-54555 add fname=</mysql/mysqldump.sql> lstat=A A IHA A A A
> A -B BAA B BWoVsJ BWoVsJ BWoVsJ A A E delta_seq=0 chksum=
> fd_plugins.c:329-54555 plugin cmd=bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> fd_plugins.c:505-54555 plugin=bpipe-fd.so plen=5
> cmd=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh
> mydb:/etc/bacula/scripts/mysqldump.sh -R len=5
> fd_plugins.c:127-54555 name=bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> len=5 plugin=bpipe-fd.so plen=5
> fd_plugins.c:521-54555 Command plugin = bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> bpipe-fd.c:253-54555 bpipe-fd: pluginEvent
> cmd=bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh
> mydb:/etc/bacula/scripts/mysqldump.sh -R
> fd_plugins.c:583-54555 startBackup returned type=3,
> fname=/srv/mysqldump.sql
> backup.c:289-54555 FT_REG saving: /srv/mysqldump.sql
> backup.c:390-54555 bfiled: sending /srv/mysqldump.sql to stored
> fd_plugins.c:778-54555 send_plugin_name=bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> fd_plugins.c:802-54555 send plugin start/end: 1 1 1
> bpipe:/srv/mysqldump.sql:/bin/sh /etc/bacula/scripts/mysqldump.sh
> mydb:/etc/bacula/scripts/mysqldump.sh -R
> backup.c:1352-54555 No strip for /srv/mysqldump.sql
> bpipe-fd.c:360-54555 bpipe-fd: IO_OPEN fd=7f33e4003528 reader=/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb
> fd_plugins.c:778-54555 send_plugin_name=bpipe:/srv/mysqldump.sql:/bin/sh
> /etc/bacula/scripts/mysqldump.sh mydb:/etc/bacula/scripts/mysqldump.sh -R
> accurate.c:46-54555 </srv/mysqldump.sql> not found to be marked as seen
> accurate.c:153-54555 deleted fname=/mysql/mysqldump.sql seen=0
>
>
> As accurate.c complained about the file I disabled Accurate backup, but
> the warning remains:
>
> JobId 54556: Warning: Cannot stat file /srv/mysqldump.sql: ERR=No such
> file or directory
>
>
> --
> Silver
>
> On 22.01.2016 13:45, Silver Salonen wrote:
>
> I know how bpipe works and my method using script is to work around an
> issue with bpipe erroring out with "Error closing stream for pseudo file"
> (no idea why, but running the same mysqldump command from inside script
> solves this). Contents of the script is really straight-forward and doesn't
> change behavior of the dump being sent to STDOUT:
>
> #!/bin/sh
>
> user=backup
> pswd=PASSWORD
>
> if [ "$1" = "-R" ]; then
> /usr/bin/mysql --default-character-set=utf8 -u "$user" -p"$pswd"
> else
> /usr/bin/mysqldump --single-transaction --routines --events --opt
> --max_allowed_packet 128M --default-character-set=utf8 --log-error=
> /var/log/mysql/mysqldump_err.log -u "$user" -p"$pswd" --databases $*
> fi
>
> exit 0
>
>
> BTW, I discovered that the error "bpipe-fd: Error closing for file
> /srv/mysqldump.sql" was caused of the wrong credentials I had in the
> script. The first error still remains.
>
> --
> Silver
>
> On 22.01.2016 12:55, Heitor Faria wrote:
>
> Hello Silver,
>
> Hello Silver [2],
>
> Please read:
> http://bacula.us/using-bpipe-to-stream-dumps-vm-clones-and-another-data-to-your-backup
> bpipe reads a command data that is directed to stdout and stores in backup
> with the name you put in the 2nd variable (fifo path) => I hope is what you
> are trying to achieve with your scripts. While in restore Bacula uses the
> last bpipe configuration variable (command) to read from stdin and restore
> your data with the application of your choice. I never saw a bpipe usage
> documentation with scripts, but with commands directly. E.g.: Plugin =
> “bpipe:/mnt/fifo/heitorfaria.sql:mysqldump -u root -ppasswd –databases
> heitorfaria:mysql -u root -ppasswd″
> The beauty of bpipe is you don't need local disk space to generate the
> dumps since fifos actually don't hold data (it just streams it to an end)
> and you may also automate application restore directly to it.
> I recorded some videoclasses that you can find in the same URL post I
> provided.
>
>
>> JobId 54542: Error: /srv/mysqldump.sql mtime changed during backup.
>> JobId 54542: Error: bpipe-fd: Error closing for file /srv/mysqldump.sql:
>> 268435583
>> ...
>> Termination: Backup OK -- with warnings
>>
>> --
>> Silver
>>
>> On 22.01.2016 10:56, Ana Emília M. Arruda wrote:
>>
>> Hello Silver,
>>
>> The same error? Have you checked permissions?
>>
>> Best regards,
>> Ana
>>
>> On Fri, Jan 22, 2016 at 9:19 AM, Silver Salonen <
>> <silver.salo...@gmail.com>silver.salo...@gmail.com> wrote:
>>
>>> After creating those folders, those errors still appeared. Any other
>>> ideas?
>>>
>>> --
>>> Silver
>>>
>>> On 21.01.2016 13:43, Silver Salonen wrote:
>>>
>>> Aha, I indeed didn't have /mysql directory. Thanks a bunch! :)
>>>
>>> PS. Script permissions are correct as otherwise the dump wouldn't have
>>> been created.
>>>
>>> --
>>> Silver
>>>
>>> On 21.01.2016 12:21, Heitor Faria wrote:
>>>
>>> Hello Silver,
>>>
>>> Does the /mysql directory exist? It must.
>>> Make sure also bacula-fd daemon OS user has proper permissions to run
>>> the scripts
>>>
>>> Regards,
>>> ===========================================================================
>>>
>>> Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified
>>> Administrator II
>>> Do you need Bacula training? <http://bacula.us/video-classes>
>>> http://bacula.us/video-classes/
>>> +55 61 8268-4220
>>> Site: <http://bacula.us>http://bacula.us FB: heitor.faria
>>> ===========================================================================
>>>
>>>
>>> Enviado por TypeApp <http://www.typeapp.com/r>
>>>
>>> Em 21 de jan de 2016, pelo 06:37, Silver Salonen <
>>> <silver.salo...@gmail.com>silver.salo...@gmail.com> escrito:
>>>>
>>>> Hi.
>>>>
>>>> I'm backing up MySQL databases from several servers with Fileset line:
>>>> Plugin =
>>>> "bpipe:/mysql/mysqldump.sql:/etc/bacula/scripts/mysqldump.sh
>>>> mydb:/etc/bacula/scripts/mysqldump.sh -R"
>>>>
>>>> FD on some of those servers give the following error:
>>>> Warning: Cannot stat file /mysql/mysqldump.sql: ERR=No such file or
>>>> directory
>>>>
>>>> I've confirmed that actually the MySQL database is still backed up and
>>>> can be restored too. So why the error? Anybody knows or can guess?
>>>>
>>>> --
>>>> Silver
>>>>
>>>>
> Regards,
> --
> =======================================================================
> Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems Certified
> Administrator II
> Próximas aulas telepresencial ao-vivo - 15 de fevereiro:
> http://www.bacula.com.br/agenda/
> Ministro treinamento e implementação in-company Bacula:
> http://www.bacula.com.br/in-company/
> Ou assista minhas vídeo aulas on-line:
> <http://www.bacula.com.br/treinamento-bacula-ed/>
> http://www.bacula.com.br/treinamento-bacula-ed/
> 61 <%2B55%2061%202021-8260>8268-4220 <%2B55%2061%208268-4220>
> Site: www.bacula.com.br | Facebook: heitor.faria
> <http://www.facebook.com/heitor.faria>
> ========================================================================
>
>
>
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
