Keep in mind that bpipe simply "executes" the "program" on the other
end, which means that if you execute a binary it does not run under the
shell. If you put the execution of the binary inside a file then
execute it, normally you have a #!/bin/sh at the top of the file and
hence the binary is executed under a shell.
This could make a difference if something in your plugin command
contains shell characters ...
Best regards,
Kern
On 01/25/2016 12:42 AM, Silver Salonen wrote:
> Yeah, so what can we conclude from this now? :)
>
> --
> Silver
>
> On 22.01.2016 17:55, Wanderlei Huttel wrote:
>> Hi Silver
>>
>> I've made a test using your script and worked fine!
>> I'm using bacula 7.4.0.
>>
>> - MySQL user has all privileges;
>> grant all on *.* to 'user'@'localhost' identified by 'password';
>> - The directory and file /mnt/fifo/intranet.sql doesn't exists
>> - Permission of script is 775
>>
>> ##################################################
>> # FileSet teste bpipe
>> FileSet {
>> Name = "FileSet_Bpipe_Bacula"
>> Include {
>> Options {
>> signature = md5
>> compression = gzip
>> }
>> Plugin =
>> "bpipe:/backup/dump_mysql.sql:/usr/local/bin/backup_mysql.sh
>> intranet:/usr/local/bin/backup_mysql.sh -R"
>> }
>> }
>>
>> ##################################################
>>
>>
>>
>> Backup Log
>> 2016-01-22 13:43:01 bacula-dir JobId 3012: Start Backup JobId 3012,
>> Job=Backup_Bpipe_Bacula.2016-01-22_13.42.59_48
>> 2016-01-22 13:43:01 bacula-dir JobId 3012: Using Device "DiscoLocal1" to
>> write.
>> 2016-01-22 13:43:01 bacula-sd JobId 3012: Volume "Volume-Default-0001"
>> previously written, moving to end of data.
>> 2016-01-22 13:43:01 bacula-sd JobId 3012: Ready to append to end of Volume
>> "Volume-Default-0001" size=35,757,662
>> 2016-01-22 13:43:05 bacula-sd JobId 3012: Elapsed time=00:00:04, Transfer
>> rate=1.680 M Bytes/second
>> 2016-01-22 13:43:05 bacula-sd JobId 3012: Sending spooled attrs to the
>> Director. Despooling 285 bytes ...
>> 2016-01-22 13:43:05 bacula-dir JobId 3012: Bacula bacula-dir 7.4.0
>> (16Jan16):
>> Build OS: x86_64-unknown-linux-gnu debian 8.2
>> JobId: 3012
>> Job: Backup_Bpipe_Bacula.2016-01-22_13.42.59_48
>> Backup Level: Full
>> Client: "srv_bacula-fd" 7.4.0 (16Jan16)
>> x86_64-unknown-linux-gnu,debian,8.2
>> FileSet: "FileSet_Bpipe_Bacula" 2016-01-22 13:42:59
>> Pool: "Pool-Default" (From Job resource)
>> Catalog: "MyCatalog" (From Client resource)
>> Storage: "StorageLocal1" (From Pool resource)
>> Scheduled time: 22-Jan-2016 13:42:59
>> Start time: 22-Jan-2016 13:43:01
>> End time: 22-Jan-2016 13:43:05
>> Elapsed time: 4 secs
>> Priority: 10
>> FD Files Written: 1
>> SD Files Written: 1
>> FD Bytes Written: 6,722,331 (6.722 MB)
>> SD Bytes Written: 6,722,542 (6.722 MB)
>> Rate: 1680.6 KB/s
>> Software Compression: 85.1% 6.7:1
>> Snapshot/VSS: no
>> Encryption: no
>> Accurate: no
>> Volume name(s): Volume-Default-0001
>> Volume Session Id: 7
>> Volume Session Time: 1453470218
>> Last Volume Bytes: 42,492,736 (42.49 MB)
>> Non-fatal FD errors: 0
>> SD Errors: 0
>> FD termination status: OK
>> SD termination status: OK
>> Termination: Backup OK
>> 2016-01-22 13:43:05 bacula-dir JobId 3012: Begin pruning Jobs older than 1
>> year .
>> 2016-01-22 13:43:05 bacula-dir JobId 3012: No Jobs found to prune.
>> 2016-01-22 13:43:05 bacula-dir JobId 3012: Begin pruning Files.
>> 2016-01-22 13:43:05 bacula-dir JobId 3012: No Files found to prune.
>> 2016-01-22 13:43:05 bacula-dir JobId 3012: End auto prune.
>> 2016-01-22 13:43:05 bacula-dir JobId 3012: shell command: run AfterJob
>> "/etc/bacula/scripts/send_whatsapp.sh "Client=srv_bacula-fd" "JobId=3012"
>> "JobName=Backup_Bpipe_Bacula" "JobBytes=6722331" "JobFiles=1" "Level=Full"
>> "Pool=Pool-Default" "Storage=StorageLocal1" "Status=OK""
>> 2016-01-22 13:43:06 bacula-dir JobId 3012: AfterJob:
>> INFO:yowsup.demos.sendclient.layer:Message sent
>>
>>
>>
>> Restore Log
>> 2016-01-22 13:44:44 bacula-dir JobId 3013: Start Restore Job
>> Restore_Files.2016-01-22_13.44.42_56
>> 2016-01-22 13:44:44 bacula-dir JobId 3013: Using Device "DiscoLocal1" to
>> read.
>> 2016-01-22 13:44:44 bacula-sd JobId 3013: Ready to read from volume
>> "Volume-Default-0001" on file device "DiscoLocal1" (/mnt/disco_usb).
>> 2016-01-22 13:44:44 bacula-sd JobId 3013: Forward spacing Volume
>> "Volume-Default-0001" to file:block 0:35757662.
>> 2016-01-22 13:44:45 bacula-sd JobId 3013: End of Volume at file 0 on
>> device "DiscoLocal1" (/mnt/disco_usb), Volume "Volume-Default-0001"
>> 2016-01-22 13:44:45 bacula-sd JobId 3013: End of all volumes.
>> 2016-01-22 13:44:45 bacula-sd JobId 3013: Elapsed time=00:00:01, Transfer
>> rate=6.722 M Bytes/second
>> 2016-01-22 13:45:57 bacula-dir JobId 3013: Bacula bacula-dir 7.4.0
>> (16Jan16):
>> Build OS: x86_64-unknown-linux-gnu debian 8.2
>> JobId: 3013
>> Job: Restore_Files.2016-01-22_13.44.42_56
>> Restore Client: srv_bacula-fd
>> Start time: 22-Jan-2016 13:44:44
>> End time: 22-Jan-2016 13:45:57
>> Files Expected: 1
>> Files Restored: 1
>> Bytes Restored: 45,185,460
>> Rate: 619.0 KB/s
>> FD Errors: 0
>> FD termination status: OK
>> SD termination status: OK
>> Termination: Restore OK
>> 2016-01-22 13:45:57 bacula-dir JobId 3013: Begin pruning Jobs older than 1
>> year .
>> 2016-01-22 13:45:57 bacula-dir JobId 3013: No Jobs found to prune.
>> 2016-01-22 13:45:57 bacula-dir JobId 3013: Begin pruning Files.
>> 2016-01-22 13:45:57 bacula-dir JobId 3013: No Files found to prune.
>> 2016-01-22 13:45:57 bacula-dir JobId 3013: End auto prune.
>> Best Regards
>> Wanderlei
>>
>> 2016-01-22 11:21 GMT-02:00 Silver Salonen <silver.salo...@gmail.com
>> <mailto:silver.salo...@gmail.com>>:
>>
>> Debug output concerning mysqldump from /opt/bacula/bin/bacula-fd
>> -d 200 -c bacula-fd.conf -f
>>
>>> job.c:1227-54555 P bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> fd_plugins.c:329-54555 plugin
>>> cmd=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> fd_plugins.c:127-54555 name=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R len=5
>>> plugin=bpipe-fd.so plen=5
>>> bpipe-fd.c:215-54555 bpipe-fd:
>>> PluginCommand=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> accurate.c:230-54555 add fname=</mysql/mysqldump.sql> lstat=A A
>>> IHA A A A A -B BAA B BWoVsJ BWoVsJ BWoVsJ A A E delta_seq=0 chksum=
>>> fd_plugins.c:329-54555 plugin
>>> cmd=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> fd_plugins.c:505-54555 plugin=bpipe-fd.so plen=5
>>> cmd=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R len=5
>>> fd_plugins.c:127-54555 name=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R len=5
>>> plugin=bpipe-fd.so plen=5
>>> fd_plugins.c:521-54555 Command plugin =
>>> bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> bpipe-fd.c:253-54555 bpipe-fd: pluginEvent
>>> cmd=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> fd_plugins.c:583-54555 startBackup returned type=3,
>>> fname=/srv/mysqldump.sql
>>> backup.c:289-54555 FT_REG saving: /srv/mysqldump.sql
>>> backup.c:390-54555 bfiled: sending /srv/mysqldump.sql to stored
>>> fd_plugins.c:778-54555
>>> send_plugin_name=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> fd_plugins.c:802-54555 send plugin start/end: 1 1 1
>>> bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> backup.c:1352-54555 No strip for /srv/mysqldump.sql
>>> bpipe-fd.c:360-54555 bpipe-fd: IO_OPEN fd=7f33e4003528
>>> reader=/bin/sh /etc/bacula/scripts/mysqldump.sh mydb
>>> fd_plugins.c:778-54555
>>> send_plugin_name=bpipe:/srv/mysqldump.sql:/bin/sh
>>> /etc/bacula/scripts/mysqldump.sh
>>> mydb:/etc/bacula/scripts/mysqldump.sh -R
>>> accurate.c:46-54555 </srv/mysqldump.sql> not found to be marked
>>> as seen
>>> accurate.c:153-54555 deleted fname=/mysql/mysqldump.sql seen=0
>>
>> As accurate.c complained about the file I disabled Accurate
>> backup, but the warning remains:
>>> JobId 54556: Warning: Cannot stat file /srv/mysqldump.sql:
>>> ERR=No such file or directory
>>
>> --
>> Silver
>>
>> On 22.01.2016 13:45, Silver Salonen wrote:
>>> I know how bpipe works and my method using script is to work
>>> around an issue with bpipe erroring out with "Error closing
>>> stream for pseudo file" (no idea why, but running the same
>>> mysqldump command from inside script solves this). Contents of
>>> the script is really straight-forward and doesn't change
>>> behavior of the dump being sent to STDOUT:
>>>
>>>> |#!/bin/sh|
>>>>
>>>> |user=backup|
>>>> |pswd=PASSWORD|
>>>>
>>>> |if| |[ ||"$1"| |= ||"-R"| |]; ||then|
>>>> | ||/usr/bin/mysql| |--default-character-||set||=utf8
>>>> -u ||"$user"| |-p||"$pswd"|
>>>> |else|
>>>> | ||/usr/bin/mysqldump| |--single-transaction --routines
>>>> --events --opt --max_allowed_packet 128M
>>>> --default-character-||set||=utf8
>>>> --log-error=||/var/log/mysql/mysqldump_err||.log
>>>> -u ||"$user"| |-p||"$pswd"| |--databases $*|
>>>> |fi|
>>>>
>>>> |exit| |0|
>>>
>>> BTW, I discovered that the error "bpipe-fd: Error closing for
>>> file /srv/mysqldump.sql" was caused of the wrong credentials I
>>> had in the script. The first error still remains.
>>>
>>> --
>>> Silver
>>>
>>> On 22.01.2016 12:55, Heitor Faria wrote:
>>>>
>>>> Hello Silver,
>>>>
>>>> Hello Silver [2],
>>>>
>>>> Please
>>>> read:
>>>> http://bacula.us/using-bpipe-to-stream-dumps-vm-clones-and-another-data-to-your-backup
>>>> bpipe reads a command data that is directed to stdout and
>>>> stores in backup with the name you put in the 2nd variable
>>>> (fifo path) => I hope is what you are trying to achieve with
>>>> your scripts. While in restore Bacula uses the last bpipe
>>>> configuration variable (command) to read from stdin and restore
>>>> your data with the application of your choice. I never saw a
>>>> bpipe usage documentation with scripts, but with commands
>>>> directly. E.g.: Plugin =
>>>> “bpipe:/mnt/fifo/heitorfaria.sql:mysqldump -u root -ppasswd
>>>> –databases heitorfaria:mysql -u root -ppasswd″
>>>> The beauty of bpipe is you don't need local disk space to
>>>> generate the dumps since fifos actually don't hold data (it
>>>> just streams it to an end) and you may also automate
>>>> application restore directly to it.
>>>> I recorded some videoclasses that you can find in the same URL
>>>> post I provided.
>>>>
>>>>
>>>> JobId 54542: Error: /srv/mysqldump.sql mtime changed during
>>>> backup.
>>>> JobId 54542: Error: bpipe-fd: Error closing for file
>>>> /srv/mysqldump.sql: 268435583
>>>> ...
>>>> Termination: Backup OK -- with warnings
>>>>
>>>> --
>>>> Silver
>>>>
>>>> On 22.01.2016 10:56, Ana Emília M. Arruda wrote:
>>>>
>>>> Hello Silver,
>>>>
>>>> The same error? Have you checked permissions?
>>>>
>>>> Best regards,
>>>> Ana
>>>>
>>>> On Fri, Jan 22, 2016 at 9:19 AM, Silver Salonen
>>>> <silver.salo...@gmail.com> wrote:
>>>>
>>>> After creating those folders, those errors
>>>> still appeared. Any other ideas?
>>>>
>>>> --
>>>> Silver
>>>>
>>>> On 21.01.2016 13:43, Silver Salonen wrote:
>>>>
>>>> Aha, I indeed didn't have /mysql directory.
>>>> Thanks a bunch! :)
>>>>
>>>> PS. Script permissions are correct as
>>>> otherwise the dump wouldn't have been created.
>>>>
>>>> --
>>>> Silver
>>>>
>>>> On 21.01.2016 12:21, Heitor Faria wrote:
>>>>
>>>> Hello Silver,
>>>>
>>>> Does the /mysql directory exist? It must.
>>>> Make sure also bacula-fd daemon OS user
>>>> has proper permissions to run the scripts
>>>>
>>>> Regards,
>>>>
>>>> ===========================================================================
>>>>
>>>> Heitor Medrado de Faria - LPIC-III |
>>>> ITIL-F | Bacula Systems Certified
>>>> Administrator II
>>>> Do you need Bacula training?
>>>> http://bacula.us/video-classes/
>>>> +55 61 8268-4220
>>>> <tel:%2B55%2061%208268-4220>
>>>> Site: http://bacula.us FB: heitor.faria
>>>>
>>>> ===========================================================================
>>>>
>>>>
>>>> Enviado por TypeApp
>>>> <http://www.typeapp.com/r>
>>>>
>>>> Em 21 de jan de 2016, pelo 06:37,
>>>> Silver Salonen
>>>> <silver.salo...@gmail.com> escrito:
>>>>
>>>> Hi.
>>>>
>>>> I'm backing up MySQL databases from
>>>> several servers with Fileset line:
>>>> Plugin =
>>>>
>>>> "bpipe:/mysql/mysqldump.sql:/etc/bacula/scripts/mysqldump.sh
>>>> <http://mysqldump.sh>
>>>> mydb:/etc/bacula/scripts/mysqldump.sh
>>>> <http://mysqldump.sh> -R"
>>>>
>>>> FD on some of those servers give the
>>>> following error:
>>>> Warning: Cannot stat file
>>>> /mysql/mysqldump.sql: ERR=No such file or
>>>> directory
>>>>
>>>> I've confirmed that actually the MySQL
>>>> database is still backed up and
>>>> can be restored too. So why the error?
>>>> Anybody knows or can guess?
>>>>
>>>> --
>>>> Silver
>>>>
>>>>
>>>> Regards,
>>>> --
>>>> =======================================================================
>>>> Heitor Medrado de Faria - LPIC-III | ITIL-F | Bacula Systems
>>>> Certified Administrator II
>>>> Próximas aulas telepresencial ao-vivo - 15 de
>>>> fevereiro: http://www.bacula.com.br/agenda/
>>>> Ministro treinamento e implementação in-company
>>>> Bacula: http://www.bacula.com.br/in-company/
>>>> Ou assista minhas vídeo aulas
>>>> on-line: http://www.bacula.com.br/treinamento-bacula-ed/
>>>> 61 <tel:%2B55%2061%202021-8260>8268-4220
>>>> <tel:%2B55%2061%208268-4220>
>>>> Site: www.bacula.com.br
>>>> <http://www.bacula.com.br/> | Facebook: heitor.faria
>>>> <http://www.facebook.com/heitor.faria>
>>>>
>>>> ========================================================================
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Site24x7 APM Insight: Get Deep Visibility into Application
>> Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
>> _______________________________________________
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> <mailto:Bacula-users@lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>>
>
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
>
>
> _______________________________________________
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
