Hello,
On 24.09.2005 19:25, Baldur Norddahl wrote:
Hi,
I switched from Sqlite to Postgresql. Performance was horrible until I
turned fsync off.
I suspect that bacula is inserting data about each file as a seperate
transaction.
As far as I know, you're right.
Something like this:
begin;
insert into file values (...);
commit;
Or actually it is probably just running with autocommit on, which is the
same thing.
What it should be doing, is inserting large amount of files in a single
transaction. Like this;
That also is correct. Theoretically :-)
begin;
insert into file values (...);
insert into file values (...);
insert into file values (...);
insert into file values (...);
(many more)
commit;
That will speed everything up by a huge amount. And it wouldn't be
neccessary to turn fsync off.
Hmm I also notice that there are no foreign keys. Adding them wouldn't
help speeding things up though.
Is there something in bacula that prevents running things in larger
transactions like this?
Well, this has been discussed before, and perhaps you can find the
threads in the list archives. My summary of the discussions I remember
follows, although I leave out many details and perhaps got everything
wrong...
The problem is that bacula, due to some internal limitations, has to
keep its database interaction consistent itself and can't rely on the
database doing that. So, there must not be multiple threads storing data
simultaneously, which is one reason why catalog writing can block bacula
for a while. Then - more closely related with transactions - it's quite
difficult to separate all catalog communications into transactions
correctly, because even though it would speed things up, there are some
non-vital queries that might fail, and others, (especially catalog
inserts, of course) must not be affected by these. Finally, transactions
are not easily portable to all suported database backends, so they might
require many more modifications to the unified backend layer.
All in all, transactions are theoretically possible, would probably help
a lot, but are very hard to implement. Kern once pointed out that he
would only accept a patch concerning transactions if he could be
*really* sure it didn't break anything, and the submitter would have to
more-or-less prove that, because Kern himself were quite sure that he
couldn't verify that due to the complexity of the existing code. (The
semi-quote IIRC and "please correct me, Kern", of course).
In other words, if you wanted to implement transactions for pgSQL you'd
be facing a lot of work, and not much help. And, in the end, it's easily
possible that your patch wouldn't be accepted.
Possible, but not a great chance of being implemented soon, I think.
Arno
Baldur
-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
--
IT-Service Lehmann [EMAIL PROTECTED]
Arno Lehmann http://www.its-lehmann.de
-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server.
Download it for free - -and be entered to win a 42" plasma tv or your very
own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
