Hello Timothé, Thanks for providing all details about this restore problem.
I did tests on a set of 150K files with old Bacula 9.0.6 and the latest 15.0.2. It looks that something has changed in Bacula MySQL/MariaDB support in a version between because on the version 9 it works well, while on 15 it hangs on the restore commands. Your restore using bconsole can work much better because it does not use Bacula Bvfs interface. I will try to look at it deeper in the next few days. I will let know if I find something. Best regards, Marcin Haba (gani) On Thu, 4 Apr 2024 at 16:14, timothe pennec <timit...@hotmail.fr> wrote: > Hi, > > I'm using mariadb for database and it's him seems to be slow : > top result : > > top - 16:08:51 up 2 days, 20:10, 6 users, load average: 0,97, 0,60, 0,29 > Tasks: 307 total, 1 running, 306 sleeping, 0 stopped, 0 zombie > %Cpu(s): 12,5 us, 0,0 sy, 0,0 ni, 87,3 id, 0,0 wa, 0,0 hi, 0,2 si, > 0,0 st > MiB Mem : 7936,5 total, 177,7 free, 592,8 used, 7166,1 buff/cache > MiB Swap: 4096,0 total, 4095,0 free, 1,0 used. 7026,5 avail Mem > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 29267 mysql 20 0 1346196 142604 23396 S 100,0 1,8 2:53.58 > mariadbd > > I've 8 vCPU but only one is at 100 % (can see it with htop) all other at 0. > > I've enabled debug level 500 and i can see many log right after submit > button "Run" and stuck at : > > bacula-dir: ua_restore.c:2015-0 JobId=4792 JobIds= FileIndex=10 > [All 53000 files ...] > bacula-dir: ua_restore.c:2015-0 JobId=4792 JobIds= FileIndex=53895 > bacula-dir: ua_restore.c:2015-0 JobId=4792 JobIds= FileIndex=53896 > bacula-dir: mysql.c:584-0 db_sql_query finished > bacula-dir: ua_restore.c:1476-0 query=SELECT Path.Path, File.Filename FROM > File JOIN Path USING (PathId) JOIN b210091 AS T ON (File.JobId = T.JobId > AND File.FileIndex = T.FileIndex) WHERE File.Filename LIKE > ':component_info_%' AND File.JobId IN (4792) > bacula-dir: mysql.c:527-0 db_sql_query starts with SELECT Path.Path, > File.Filename FROM File JOIN Path USING (PathId) JOIN b210091 AS T ON > (File.JobId = T.JobId AND File.FileIndex = T.FileIndex) WHERE File.Filename > LIKE ':component_info_%' AND File.JobId IN (4792) > > > Any idea ? > > Thank you > Timothé > > ------------------------------ > *De :* Marcin Haba <ganius...@gmail.com> > *Envoyé :* mercredi 3 avril 2024 21:53 > *À :* timothe pennec <timit...@hotmail.fr> > *Cc :* bacula-users@lists.sourceforge.net < > bacula-users@lists.sourceforge.net> > *Objet :* Re: [Bacula-users] [Bacularis/Baculum] Restore with web doesn't > create job before 1 hour > > Hello Timothé, > > Thanks for your detailed description of the problem. > > For start I would propose to check what is going inside Bacula and the > Catalog when this restore is trying to start. > > Yes, observing 'ps aux' is a good idea, that you did. Also useful can be > 'top' command with watching what process takes the most resources. > > From the Bacula side you can set debug for Director by this bconsole > command: > > setdebug level=500 tags=bvfs,sql trace=1 dir > > in the Bacula Director working directory you will see the > youdirector-dir.trace file where this debug lines will be written. > > If your Catalog database is PostgreSQL you ran run in the time of the > restore hanging this SQL query to see currently pending SQL queries: > > SELECT * FROM pg_stat_activity; > > Please let us know about the results. > > Good luck. > > Best regards, > Marcin Haba (gani) > > On Wed, 3 Apr 2024 at 20:40, timothe pennec <timit...@hotmail.fr> wrote: > > Hi everyone, > > I've one issue with Bacularis (and same with Baculum) WEB when i follow > the restore wizzard : > > - I select my client and let checked "Enable restore from copy job > feature" > - I select the Last full Backup of my client > - I select one folder with add button (total of files about 55 000 and > total syse about 4Go) > - I select the same client and "/tm/restore" folder for testing > - I let default setting for Restore job (RestoreFiles and no replace > file and no relocate files) > - I submit with "Run restore" > > > When i submit nothing happen (i got only "loading" favicon and not > redirect or validation message for create job). > At the same time if i open new tab to default web page it seems the server > not responding for a while and after a time the page loaded for the default > tab and error "504 Gateway timeout" for restore tab > If i ssh and launch bconsole and enter "status director" i can see that no > job was created (terminated or running or whatever state). > > Here a "ps aux | grep bconsole" at this time : > www-data 21059 0.0 0.0 2892 964 ? S 19:45 0:00 sh -c > sudo /usr/bin/bconsole -c "/etc/bacula/bconsole.conf" -D bacula-dir 2>&1 > <<END_OF_DATA gui on restore client="www.example.org" file="?b210091" > replace="never" restorejob="RestoreFiles" where="/tmp/restore" yes quit > END_OF_DATA > root 21060 0.0 0.0 8488 5228 ? S 19:45 0:00 sudo > /usr/bin/bconsole -c /etc/bacula/bconsole.conf -D bacula-dir > root 21061 0.0 0.1 165928 10516 ? Sl 19:45 0:00 > /usr/bin/bconsole -c /etc/bacula/bconsole.conf -D bacula-dir > > I'm running with bacula 15.0.2 with apt for server and clients all in > Ubuntu 22.04 > The result of "tail -f /opt/bacula/log/bacula.log" is empty for all > process. > > For about 50min / 70 min nothing happen then i got job created ! > > For exemple the log for the test run at 17:03 (job created at 18:00 and > all file copied in < 1 min) > > bacula-dir JobId 4831: End auto prune. > bacula-dir JobId 4831: No Files found to prune. > bacula-dir JobId 4831: Begin pruning Files. > bacula-dir JobId 4831: No Jobs found to prune. > bacula-dir JobId 4831: Begin pruning Jobs older than 6 months . > bacula-dir JobId 4831: Bacula bacula-dir 15.0.2 (21Mar24): > Build OS: x86_64-pc-linux-gnu-bacula ubuntu 22.04 > JobId: 4831 > Job: RestoreFiles.2024-04-02_18.00.36_48 > Restore Client: "www.example.org" 15.0.2 (21Mar24) > x86_64-pc-linux-gnu-bacula,ubuntu,22.04 > Where: /tmp/restore > Replace: Never > Start time: 02-avril-2024 18:00:38 > End time: 02-avril-2024 18:01:19 > Elapsed time: 41 secs > Files Expected: 53,888 > Files Restored: 53,888 > Bytes Restored: 4,066,211,495 (4.066 GB) > Rate: 99175.9 KB/s > FD Errors: 0 > FD termination status: OK > SD termination status: OK > Termination: Restore OK > bacula-sd JobId 4831: Elapsed time=00:00:41, Transfer rate=99.38 M > Bytes/second > bacula-sd JobId 4831: Forward spacing Volume "Monthlyvol-0017" to > addr=22149617617 > bacula-sd JobId 4831: Ready to read from volume "Monthlyvol-0017" on File > device "FileStorage" (/backup). > wwwexampleorg-fd JobId 4831: Connected to Storage at > bacula.esx2.example.org:9103 with TLS > bacula-dir JobId 4831: Connected to Client "www.example.org" at > 2001:41d0:303:f006::3:9102 with TLS > bacula-dir JobId 4831: Using Device "FileStorage" to read. > bacula-dir JobId 4831: Connected to Storage > "bacula.esx2.example.org_backup" at bacula.esx2.example.org:9103 with TLS > bacula-dir JobId 4831: Restoring files from JobId(s) 4792 > bacula-dir JobId 4831: Start Restore Job > RestoreFiles.2024-04-02_18.00.36_48 > > If i try to run restore by the bconsole command and follow the prompt job > is created instantaly. > > I didn't have test by the past the restore feature but i think something > is wrong. > > Anyone can help me for this issue ? > > Best regards > Timothé > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > > > > -- > > "Greater love hath no man than this, that a man lay down his life for his > friends." Jesus Christ > > "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za > przyjaciół swoich." Jezus Chrystus > > -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
