Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-09 Thread Dan Langille
On 8/9/2010 8:16 PM, Dan Langille wrote: > On 8/9/2010 4:23 PM, Rory Campbell-Lange wrote: >> On 04/08/10, Rory Campbell-Lange (r...@campbell-lange.net) wrote: >>> On 03/08/10, Dan Langille (d...@langille.org) wrote: On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote: >>> > Yes, a batch insert

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-09 Thread Dan Langille
On 8/9/2010 4:23 PM, Rory Campbell-Lange wrote: > On 04/08/10, Rory Campbell-Lange (r...@campbell-lange.net) wrote: >> On 03/08/10, Dan Langille (d...@langille.org) wrote: >>> On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote: >> Yes, a batch insert is faster than a specfic insert, but the latter

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-09 Thread Rory Campbell-Lange
On 04/08/10, Rory Campbell-Lange (r...@campbell-lange.net) wrote: > On 03/08/10, Dan Langille (d...@langille.org) wrote: > > On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote: > > > > Yes, a batch insert is faster than a specfic insert, but the latter > > > should be done at the "written-to-tape" tra

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-04 Thread Dan Langille
On 8/4/2010 6:50 AM, Rory Campbell-Lange wrote: > On 04/08/10, Dan Langille (d...@langille.org) wrote: > >> I think stored procedures would be a good idea. That way, we could >> improve the functionality without changing the application. >> >> That said, we do not at present use stored procedures.

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-04 Thread Rory Campbell-Lange
On 04/08/10, Dan Langille (d...@langille.org) wrote: > I think stored procedures would be a good idea. That way, we could > improve the functionality without changing the application. > > That said, we do not at present use stored procedures. We'd have to > proceed carefully. Fair enough. Can

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-04 Thread Dan Langille
On 8/4/2010 6:28 AM, Rory Campbell-Lange wrote: > On 03/08/10, Dan Langille (d...@langille.org) wrote: >> On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote: > >>> Yes, a batch insert is faster than a specfic insert, but the latter >>> should be done at the "written-to-tape" transaction time, and could

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-04 Thread Rory Campbell-Lange
On 04/08/10, Marc Cousin (cousinm...@gmail.com) wrote: > > Changes to one part of Bacula has to be compatible with all other parts > > of Bacula. For example, given that we support SQLite, PostgreSQL, and > > MySQL, we have to keep each in mind. Yes, it's a compromise. > Moreover, yes, Postgr

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-04 Thread Rory Campbell-Lange
On 03/08/10, Dan Langille (d...@langille.org) wrote: > On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote: > > Yes, a batch insert is faster than a specfic insert, but the latter > > should be done at the "written-to-tape" transaction time, and could be > > done asynchronously, but in a transaction. >

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-03 Thread Dan Langille
On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote: > Yes, a batch insert is faster than a specfic insert, but the latter > should be done at the "written-to-tape" transaction time, and could be > done asynchronously, but in a transaction. Its pretty crazy for a>7TB > tape backup to fail because of a

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-03 Thread Dan Langille
On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote: > Actually, this is what I don't get. Postgresql is a highly scalable, > robust database system and it is being used as a data dump rather than a > working tool for creating a transaction-based working catalogue. Changes to one part of Bacula has to

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-03 Thread Rory Campbell-Lange
On 03/08/10, Marc Cousin (cousinm...@gmail.com) wrote: > > > > 3. Why is Bacula using a batch file at all? Why not simply do a straight > > > >insert? > > > > > > Because 7,643,966 inserts would be much slower. > > > > Really? I've logged Bacula's performance on the server and the inserts > >

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-03 Thread Rory Campbell-Lange
On 03/08/10, Martin Simmons (mar...@lispworks.com) wrote: > > > > 2. Can I stop this needless after-backup insertion? I tried setting > > > >Spool Attributes to NO but it did not work > > > > > > You need to rebuild Bacula with the --disable-batch-insert option, but it > > > might run quite sl

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-03 Thread Martin Simmons
> On Tue, 3 Aug 2010 13:17:25 +0100, Rory Campbell-Lange said: > > Thanks very much for your response, Martin. > > On 03/08/10, Martin Simmons (mar...@lispworks.com) wrote: > > > On Tue, 3 Aug 2010 10:15:18 +0100, Rory Campbell-Lange said: > > > > I have 3.4GB free in /var where Postgres

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-03 Thread Rory Campbell-Lange
Thanks very much for your response, Martin. On 03/08/10, Martin Simmons (mar...@lispworks.com) wrote: > > On Tue, 3 Aug 2010 10:15:18 +0100, Rory Campbell-Lange said: > > I have 3.4GB free in /var where Postgresql is located. At the end of a > > large backup job (7,643,966 files taking up 7.2

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-03 Thread Martin Simmons
> On Tue, 3 Aug 2010 10:15:18 +0100, Rory Campbell-Lange said: > > I'm fairly desperate for some advice on this issue. > > I have 3.4GB free in /var where Postgresql is located. At the end of a > large backup job (7,643,966 files taking up 7.265TB of space) Postgres > bails out copying a batc

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-03 Thread Rory Campbell-Lange
I'm fairly desperate for some advice on this issue. I have 3.4GB free in /var where Postgresql is located. At the end of a large backup job (7,643,966 files taking up 7.265TB of space) Postgres bails out copying a batch file into the File table due to a mysterious "no space left on device" error.

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-02 Thread Rory Campbell-Lange
On 30/07/10, Dan Langille (d...@langille.org) wrote: > On 7/30/2010 3:53 AM, Rory Campbell-Lange wrote: > > > > Fatal error: sql_create.c:843 Batch end postgresql.c:748 error ending > > batch mode: ERROR: could not extend relation 1663/17472/17828: > > wrote only 4096 of 8192 bytes at

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-07-30 Thread Rory Campbell-Lange
Thanks very much for your response, Eric. On 30/07/10, Eric Bollengier (eric.bolleng...@baculasystems.com) wrote: > It depends on how many files you backup, but a Bacula catalog requires some > space, specially if you handle many files. I think you need help to configure > your PostgreSQL catalo

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-07-30 Thread Dan Langille
On 7/30/2010 8:22 AM, Dan Langille wrote: > Look at the Spool Attributes directive. Set it to know. know? Try no. See http://www.bacula.org/5.0.x-manuals/en/main/main/Configuring_Director.html -- Dan Langille - http://langille.org/ -

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-07-30 Thread Dan Langille
On 7/30/2010 3:53 AM, Rory Campbell-Lange wrote: > Bacula has bailed out near the end of a 6.5TB backup (which is really > frustrating!) > > Fatal error: sql_create.c:843 Batch end postgresql.c:748 error ending > batch mode: ERROR: could not extend relation 1663/17472/17828: > wrote

[Bacula-users] Bacula + Postgres : copy batch problem

2010-07-30 Thread Rory Campbell-Lange
Bacula has bailed out near the end of a 6.5TB backup (which is really frustrating!) Fatal error: sql_create.c:843 Batch end postgresql.c:748 error ending batch mode: ERROR: could not extend relation 1663/17472/17828: wrote only 4096 of 8192 bytes at block 98374 HINT: Check free d