Hi Bruno, we increased the debug level but the load on the database decreased so it ran successful. Will come back to this when there is more load and the problem happens again.
Another thing I found is that Increments seem to not work as I would expect them: The Full which ended at 2023-09-19 09:32:39 contains: /backupserver/postgresdumps/wal_archive/ /backupserver/postgresdumps/wal_archive/0000000100000135000000A1 /backupserver/postgresdumps/wal_archive/0000000100000135000000A2 /backupserver/postgresdumps/wal_archive/0000000100000135000000A2.000000 28.backup /backupserver/postgresdumps/wal_archive/0000000100000135000000A3 /backupserver/postgresdumps/wal_archive/0000000100000135000000A4 In those archive dir at 10:08 a new file 0000000100000135000000A5 was created. But the increment at(10:18:41) after that timestamp does not include the file: *list files jobid=5118974 * The log says: JobId 5118974: python3-fd-mod: Same LSN 00000135/A6517EE8 as last time - nothing to do JobId 5118974: python3-fd-mod: Current LSN 00000135/A6517EE8, last LSN: 135/A42EF7F0 JobId 5118974: python3-fd-mod: Connected to PostgreSQL version 150004 JobId 5118974: Spooling data ... JobId 5118974: ACL support is enabled JobId 5118974: Extended attribute support is enabled JobId 5118974: Connected Storage daemon at bareos:9103, encryption: None JobId 5118974: Encryption: None JobId 5118974: python3-fd-mod: Got pg major version 15 from restore object of job 5118972 JobId 5118974: python3-fd-mod: Got lastLSN 135/A42EF7F0 from restore object of job 5118972 JobId 5118974: python3-fd-mod: Got last_backup_stop_time 1695108740 from restore object of job 5118972 With kind regards, Dennis -------- Ursprüngliche Nachricht -------- Von: Bruno Friedmann (bruno-at-bareos) <[email protected]> An: bareos-users <[email protected]> Betreff: Re: [bareos-users] New PostgreSQL plugin search volunteers for testing Datum: 18.09.2023 17:39:01 Would you mind to try to run the job with a debug level set to 150 + timestamp. and attach the job log. I was trying to fix that case, and just emit a warning when the database has removed and changed a file during the backup. Le lundi 18 septembre 2023 à 17:02:23 UTC+2, Dennis Benndorf a écrit : > Hi Bruno, > > its me once again. The next run with excluded pg_subtrans dir fails > on the base directory: > > JobId 5116772: Fatal error: bareosfd: Traceback (most recent call > last): > > File "/usr/lib64/bareos/plugins/BareosFdWrapper.py", line 61, in > start_backup_file > return bareos_fd_plugin_object.start_backup_file(savepkt) > File "/usr/lib64/bareos/plugins/bareos-fd-postgresql.py", line 703, > in start_backup_file > return super().start_backup_file(savepkt) > File > "/usr/lib64/bareos/plugins/BareosFdPluginLocalFilesBaseclass.py", > line 118, in start_backup_file > mystatp.st_mode = statp.st_mode > UnboundLocalError: local variable 'statp' referenced before > assignment > > > JobId 5116772: Error: python3-fd-mod: Could net get stat-info for > file /db/pgsql/pgsql-15/base/16386/2117113: "[Errno 2] No such file > or directory: '/db/pgsql/pgsql-15/base/16386/2117113'" > > It is a active database. Any thoughts on this? > > With kind regards, > Dennis > > -------- Ursprüngliche Nachricht -------- > Von: Bruno Friedmann (bruno-at-bareos) <[email protected]> > An: bareos-users <[email protected]> > Betreff: Re: [bareos-users] New PostgreSQL plugin search volunteers > for testing > Datum: 18.09.2023 16:04:42 > > Sorry in fact it is in 15 doc too. > > Le lundi 18 septembre 2023 à 15:59:34 UTC+2, Bruno Friedmann (bruno- > at-bareos) a écrit : > > Hello Dennis, > > > > Nice you already tried the fresh PG16 meat, after checking the new > > documentation about subtrans we can see that upstream has added > > that directory as shouldn't be saved. > > > > ```The contents of the > > directories pg_dynshmem/, pg_notify/, pg_serial/, pg_snapshots/, pg > > _stat_tmp/, and pg_subtrans/ (but not the directories themselves) > > can be omitted from the backup as they will be initialized on > > postmaster startup.``` > > > > Will apply that to the plugin by default before release. > > > > Le lundi 18 septembre 2023 à 15:26:24 UTC+2, Dennis Benndorf a > > écrit : > > > Hi Bruno, > > > > > > we tested your new plugin with a quite large database with a > > > running application using it. > > > When doing a full backup we ran into: > > > > > > JobId 5116734: Fatal error: bareosfd: Traceback (most recent call > > > last): > > > File "/usr/lib64/bareos/plugins/BareosFdWrapper.py", line 61, in > > > start_backup_file > > > return bareos_fd_plugin_object.start_backup_file(savepkt) > > > File "/usr/lib64/bareos/plugins/bareos-fd-postgresql.py", line > > > 703, in start_backup_file > > > return super().start_backup_file(savepkt) > > > File > > > "/usr/lib64/bareos/plugins/BareosFdPluginLocalFilesBaseclass.py", > > > line 118, in start_backup_file > > > mystatp.st_mode = statp.st_mode > > > UnboundLocalError: local variable 'statp' referenced before > > > assignment > > > > > > JobId 5116734: Error: python3-fd-mod: Could net get stat-info for > > > file /db/pgsql/pgsql-15/pg_subtrans/00BF: "[Errno 2] No such file > > > or directory: '/db/pgsql/pgsql-15/pg_subtrans/00BF'" > > > > > > It seems that the plugin fails when it wants to backup a > > > subtransaction that has been commited and removed from that > > > directory in the meantime. > > > > > > Is it save to exclude the pg_subtrans directory? > > > > > > With kind regards, > > > Dennis > > > > > > -------- Ursprüngliche Nachricht -------- > > > Von: Bruno Friedmann <[email protected]> > > > An: bareos-users <[email protected]> > > > Betreff: [bareos-users] New PostgreSQL plugin search volunteers > > > for testing > > > Datum: 13.09.2023 09:46:08 > > > > > > Hi community, users and customers, > > > > > > As we're moving towards Bareos 23, we could use some help testing > > > the all new PostgreSQL plugin which is needed because of changes > > > happened in PostgreSQL 15. > > > > > > Main features compared to the old one: > > > - use non exclusive mode > > > - support all clusters starting from version 10 > > > - support clusters with tablespace > > > - support non standard port > > > - support password parameter > > > - better debugging > > > - renew of parameters names and variables > > > - renewed documentation > > > > > > We're looking for a few early adopters to work with directly. If > > > you'd like to participate please contact > > > https://bareos.com/contact with a small description of your > > > environment to have multiple environments as test scenarios. > > > > > > And "For all others who'd like to test without early adapters > > > program the testing packages are now available > > > at this address https://download.bareos.org/experimental/PR-1541/ > > > The corresponding documentation is also available on > > > > https://download.bareos.org/experimental/PR-1541/BareosMainReference/TasksAndConcepts/Plugins.html#postgresql-plugin > > > > > > The ongoing development PR is available here > > > https://github.com/bareos/bareos/pull/1541 > > > > > > We would really appreciate comments, remarks, and tests. > > > Better to use github comments if you want to participate, and > > > help. > > > > > > The new plugin aims to replace the old one, and will be delivered > > > in the same package > > > bareos-filedaemon-postgresql-python-plugin > > > Notice The old plugin (deprecated) will still also be delivered > > > in 23, but we recommend (not sure if we will force) its usage to > > > only restore previous backups done with, until you migrate to the > > > new one. > > > > > > Thanks and all the best > > > -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/c8072468-d608-4c1d-87a5-7bf07a15a3fbn%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/ffba1f747595fdeb54ff6070fda6cc9cf32a6368.camel%40googlemail.com.
