Re: pgsql: Non text modes for pg_dumpall, correspondingly change pg_restore

On Wed, 16 Apr 2025 at 01:52, Álvaro Herrera wrote: > > On 2025-Apr-15, Mahendra Singh Thalor wrote: > > > I took this patch and did some testing. Patch looks good to me. > > Hello, thank you very much for looking. > > > I was not able to use "git am"

Re: Non-text mode for pg_dumpall

On Sat, 5 Apr 2025 at 01:41, Andrew Dunstan wrote: > > > On 2025-04-04 Fr 5:12 AM, Mahendra Singh Thalor wrote: > > On Fri, 4 Apr 2025 at 13:52, Mahendra Singh Thalor wrote: > > On Fri, 4 Apr 2025 at 01:17, Andrew Dunstan wrote: > > On 2025-04-01 Tu 1:59 AM, Mahendra

Re: [Proposal] Add \dAt [AMPTRN [TBLPTRN]] to list tables by Table Access Method in psql

- test pg_statistic pg_type pg_toast_1255 pg_toast_1255_index pg_toast_1247 pg_toast_1247_index pg_toast_2604 pg_toast_2604_index pg_toast_2606 pg_toast_2606_index pg_toast_2610 pg_toast_2610_index pg_toast_2612 - In the above command, we can add FILT

Re: pgsql: Non text modes for pg_dumpall, correspondingly change pg_restore

/* Extract dboid. */ > sscanf(line, "%u", &db_oid); > - sscanf(line, "%20s", db_oid_str); > + > + while(isdigit(*p)) > + p++; > + > + Assert(*p == ' '); > >

use correct variable in error message in _allocAH function (pg_backup_archiver.c)

Hi hackers, In _allocAH function(pg_backup_archiver.c), we were using the *fmt* variable in switch case for *default case*, but correct variable is AH->format. Here, I am attaching a patch for the same. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

pg_restore --format= option(without any value) should report an error as pg_dump is reporting an error

added 2 TAP tests also for invalid options. *Note:* We have 2 more options in pg_restore code which validate the option if arg has non zero length. I will prepare patches for both(--host and --port). We need to add some validate function also for both these options. -- Thanks an

Re: remove unnecessary explicit type conversion (to char) for appendStringInfoChar function calls

On Sat, 12 Apr 2025 at 23:56, Andrew Dunstan wrote: > > > On 2025-04-11 Fr 1:36 PM, Mahendra Singh Thalor wrote: > > Hi, > > In the current master code, 3 places we are using appendStringInfoChar > > call with explicit type conversion into char. This is not needed as &

remove unnecessary explicit type conversion (to char) for appendStringInfoChar function calls

sions on head. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v01_remove-unnecessary-type-conversion-into-char-for-appendStringInfoChar.patch Description: Binary data

Re: merge file_exists_in_directory and _fileExistsInDirectory functions and move into common file dumputils.c

On Fri, 11 Apr 2025 at 10:21, Michael Paquier wrote: > > On Thu, Apr 10, 2025 at 10:41:33PM +0530, Mahendra Singh Thalor wrote: > > We have file_exists_in_directory function in pg_restore.c and same > > code we are using in _fileExistsInDirectory function in pg_backup_a

add some more error checks into _fileExistsInDirectory function to report "too long name" error

resql.org/message-id/202504110938.4kx73ylnv6p4%40alvherre.pgsql -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v01-add-some-more-error-checks-into-_fileExistsInDirecto.patch Description: Binary data

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

do any change to server side to fix the issue of \n\r in database names. But as a cleanup patch, we can give an alert to the user by "pg_upgrade --check". As per current code, pg_dump and pg_upgrade will fail with "shell command" error but in the attached patch, we will give s

Re: Non-text mode for pg_dumpall

On Sat, 5 Apr 2025 at 01:41, Andrew Dunstan wrote: > > > On 2025-04-04 Fr 5:12 AM, Mahendra Singh Thalor wrote: > > On Fri, 4 Apr 2025 at 13:52, Mahendra Singh Thalor wrote: > > On Fri, 4 Apr 2025 at 01:17, Andrew Dunstan wrote: > > On 2025-04-01 Tu 1:59 AM, Mahendra

Re: as per commit 643a1a61985bef2590496, move create/open dir code to function using switch case of pg_backup_directory.c file also

On Thu, 10 Apr 2025 at 21:48, Andrew Dunstan wrote: > > > On 2025-04-07 Mo 2:59 PM, Mahendra Singh Thalor wrote: > > On Mon, 7 Apr 2025 at 18:52, Andrew Dunstan wrote: > >> > >> On 2025-04-07 Mo 8:25 AM, Mahendra Singh Thalor wrote: > >>> Hi, >

merge file_exists_in_directory and _fileExistsInDirectory functions and move into common file dumputils.c

Hi, We have file_exists_in_directory function in pg_restore.c and same code we are using in _fileExistsInDirectory function in pg_backup_archiver.c also. Here, I am attaching a patch to move these duplicate functions into dumputils.c file -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB

Re: as per commit 643a1a61985bef2590496, move create/open dir code to function using switch case of pg_backup_directory.c file also

On Mon, 7 Apr 2025 at 18:52, Andrew Dunstan wrote: > > > On 2025-04-07 Mo 8:25 AM, Mahendra Singh Thalor wrote: > > Hi, > > In commit 643a1a61985bef2590496, we did some cleanup and we replaced > > if-else with switch case. > > Basically, we made a function

as per commit 643a1a61985bef2590496, move create/open dir code to function using switch case of pg_backup_directory.c file also

similar. We can move this similar function into one common file also but as of now, I made a static function same as pg_dumpall.c. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v01_move-create-dir-code-to-the-switch-in-pg_backup_dir.patch Description

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

> should > - * reject those characters in CREATE ROLE and CREATE DATABASE, because use > - * there eventually leads to errors here. > - * > * appendShellString() simply prints an error and dies if LF or CR > appears. > Cheers > > [1] https://postgr.es/m/cakytnaovkql5rld4p4hzxzsnthwo-j4q3y1vtdhqgjzwc-k...@mail.gmail.com > > -- > Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ > "Aprender sin pensar es inútil; pensar sin aprender, peligroso" (Confucio) -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Non-text mode for pg_dumpall

On Fri, 4 Apr 2025 at 01:17, Andrew Dunstan wrote: > > > On 2025-04-01 Tu 1:59 AM, Mahendra Singh Thalor wrote: > > On Mon, 31 Mar 2025 at 23:43, Álvaro Herrera > > wrote: > >> Hi > >> > >> FWIW I don't think the on_exit_nicely business is

Re: Non-text mode for pg_dumpall

On Mon, 31 Mar 2025 at 19:27, Andrew Dunstan wrote: > > > On 2025-03-31 Mo 5:34 AM, Mahendra Singh Thalor wrote: > > > > > > > > There are a couple of rough edges, though. > > > > > > First, I see this: > > > > > > > > >

Re: Non-text mode for pg_dumpall

On Fri, 4 Apr 2025 at 13:52, Mahendra Singh Thalor wrote: > > On Fri, 4 Apr 2025 at 01:17, Andrew Dunstan wrote: > > > > > > On 2025-04-01 Tu 1:59 AM, Mahendra Singh Thalor wrote: > > > On Mon, 31 Mar 2025 at 23:43, Álvaro Herrera > > > wrote: >

Re: Non-text mode for pg_dumpall

On Sun, 30 Mar 2025 at 22:20, Andrew Dunstan wrote: > > > On 2025-03-29 Sa 1:17 AM, Mahendra Singh Thalor wrote: > > On Sat, 29 Mar 2025 at 03:50, Andrew Dunstan wrote: > >> > >> On 2025-03-27 Th 5:15 PM, Andrew Dunstan wrote: > >>> On 2025-

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

Thanks Andrew for the review. On Sat, 5 Apr 2025 at 20:12, Andrew Dunstan wrote: > > > On 2025-04-02 We 7:45 AM, Mahendra Singh Thalor wrote: > > On Fri, 28 Mar 2025 at 20:13, Nathan Bossart wrote: > > > > On Fri, Mar 28, 2025 at 05:08:26PM +0530, Mahendra Singh Tha

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

On Thu, 30 Jan 2025 at 16:47, Srinath Reddy wrote: > > > > On Wed, Jan 29, 2025 at 9:55 PM Mahendra Singh Thalor wrote: >> >> Hi, >> While doing some testing with pg_dumpall, I noticed one weird behaviour. >> >> While we create the database, we are allo

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

, I am attaching an updated patch which has check in createdb and RenameDatabase. For older versions, we can add more useful error message (like: rename database as database has \n\r") I will add some TAP tests and will make patches for older branches. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v02_block-database-name-with-newline-or-carriage-return.patch Description: Binary data

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

On Fri, 28 Mar 2025 at 20:13, Nathan Bossart wrote: > > On Fri, Mar 28, 2025 at 05:08:26PM +0530, Mahendra Singh Thalor wrote: > > Here, I am attaching updated patches for review. > > > > v04_001* has the changes for CREATE DATABASE/ROLE/USER and > > v04_002* has

Re: Non-text mode for pg_dumpall

d for rest database, we will update only shutdown_info as we already closed connection for previous database.With this fix, we will not touch entry of on_exit_nicely_list for each database. Here, I am attaching updated patches. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.

Re: Non-text mode for pg_dumpall

On Sat, 29 Mar 2025 at 03:50, Andrew Dunstan wrote: > > > On 2025-03-27 Th 5:15 PM, Andrew Dunstan wrote: > > > > On 2025-03-19 We 2:41 AM, Mahendra Singh Thalor wrote: > >> On Wed, 12 Mar 2025 at 21:18, Andrew Dunstan > >> wrote: > >>> >

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

On Fri, 28 Mar 2025 at 01:37, Mahendra Singh Thalor wrote: > > On Thu, 27 Mar 2025 at 18:33, Álvaro Herrera wrote: > > > > On 2025-Mar-27, Andrew Dunstan wrote: > > > > > I don't think we can backpatch this. It's a behaviour change. > > >

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

luster. Here, I am attaching an updated patch for review. This patch has changes for: CREATE DATABASE, CREATE ROLE, CREATE USER and RENAME DATABASE/USER/ROLE and have some tests also. (EXCEPT RENAME test case) -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v03_don-t-allow-newline-or-carriage-return-in-db-user-role-names.patch Description: Binary data

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

On Thu, 27 Mar 2025 at 18:10, Andrew Dunstan wrote: > > > On 2025-03-27 Th 7:57 AM, Mahendra Singh Thalor wrote: > > On Thu, 27 Mar 2025 at 16:16, Andrew Dunstan wrote: > >> > >> On 2025-03-26 We 8:52 AM, Srinath Reddy wrote: > >> > >> sor

Re: getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

On Thu, 20 Mar 2025 at 23:39, Mahendra Singh Thalor wrote: > > On Thu, 30 Jan 2025 at 16:47, Srinath Reddy wrote: > > > > > > > > On Wed, Jan 29, 2025 at 9:55 PM Mahendra Singh Thalor < mahi6...@gmail.com> wrote: > >> > >> Hi, > >>

Re: change on_exit_nicely_list array to the dynamic array to increase slots at run time for pg_restore

On Tue, 11 Mar 2025 at 11:58, Dilip Kumar wrote: > > On Mon, Mar 10, 2025 at 2:24 PM Mahendra Singh Thalor > wrote: > > > > Hi, > > (refer file src/bin/pg_dump/pg_backup_utils.c) > > > > While doing some code changes with pg_dumpall and pg_rsetore[1], w

Re: Non-text mode for pg_dumpall

On Wed, 12 Mar 2025 at 21:18, Andrew Dunstan wrote: > > > On 2025-03-12 We 3:03 AM, jian he wrote: > > On Wed, Mar 12, 2025 at 1:06 AM Álvaro Herrera > > wrote: > >> Hello, > >> > >> On 2025-Mar-11, Mahendra Singh Thalor wrote: > >> &

Re: Non-text mode for pg_dumpall

On Tue, 11 Mar 2025 at 20:12, Álvaro Herrera wrote: > > On 2025-Mar-11, Mahendra Singh Thalor wrote: > > > On Wed, 5 Mar 2025 at 20:42, Álvaro Herrera wrote: > > > > Okay, we should probably fix that, but I think the new map.dat file your > > > patch add

Re: Non-text mode for pg_dumpall

Thanks Alvaro and Jian for the review and feedback. On Wed, 5 Mar 2025 at 20:42, Álvaro Herrera wrote: > > Disclaimer: I didn't review these patches fully. > > On 2025-Mar-05, Mahendra Singh Thalor wrote: > > > On Wed, 5 Mar 2025 at 01:02, Álvaro Herrera wrote:

change on_exit_nicely_list array to the dynamic array to increase slots at run time for pg_restore

array <202503051512.tr7vcirldfns@alvherre.pgsql> -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v01-enlarge-on_exit_nicely_list-array-whenever-no-slot.patch Description: Binary data

Re: Non-text mode for pg_dumpall

Thanks Alvaro for feedback and review. On Wed, 5 Mar 2025 at 20:42, Álvaro Herrera wrote: > > Disclaimer: I didn't review these patches fully. > > On 2025-Mar-05, Mahendra Singh Thalor wrote: > > > On Wed, 5 Mar 2025 at 01:02, Álvaro Herrera wrote: > > > >

Re: Non-text mode for pg_dumpall

w line in double quote <https://www.postgresql.org/message-id/flat/CAFC%2Bb6qwc%2Bwpt7_b2R6YhpDkrXeFvFd5NoLbTMMoxX9tfOHjpg%40mail.gmail.com#6423324382ff7a0537b2fe6231eb1e09> -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Non-text mode for pg_dumpall

minor cosmetic changes for v19. - /* return number of errors */ > - if (AH->n_errors) > - n_errors = AH->n_errors; > - > /* AH may be freed in CloseArchive? */ > CloseArchive(AH); As per this comment, we can't return AH->n_errors as this might already be free

Re: Non-text mode for pg_dumpall

attaching updated patches for review and testing. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v19_0001_move-common-code-of-pg_dumpall-and-pg_restore-to-new_file.patch Description: Binary data v19_0002_pg_dumpall-with-non-text_format-20th_feb.patch Description: Binary data

Re: Non-text mode for pg_dumpall

at the end of the restoration. > Do we need to apply this to restore executing global commands (create > role, create tablespace)? > If not then we need to put some words in pg_restoe --exit-on-error > option saying that while restoring global objects --exit-on-error > option is ignore

Re: Non-text mode for pg_dumpall

then the user should use a single database dump path. If we allow users without the --create option, then pg_restore will create all the tables under a single database even if those tables are in different databases. I will fix the -g option(1st test case) in the next patch. -- Thanks

Re: Non-text mode for pg_dumpall

Thanks Jian. On Wed, 12 Feb 2025 at 12:45, jian he wrote: > > On Wed, Feb 12, 2025 at 1:17 AM Mahendra Singh Thalor > wrote: > > > > > > > > There are some tests per https://commitfest.postgresql.org/52/5495, I > > > will check it later. > >

Re: Non-text mode for pg_dumpall

tion."); okay. > > > Bug. > when pg_restore --globals-only can be applied when we are restoring a > single database (can be an output of pg_dump). As of now, we are ignoring this option. We can add an error in the "else" part of the global.dat file. Ex: option --globals-only is only supported with dump of pg_dumpall. Similarly --exclude-database also. > > > There are some tests per https://commitfest.postgresql.org/52/5495, I > will check it later. > The attached patch is the change for the above reviews. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Non-text mode for pg_dumpall

e will still call > process_global_sql_commands. > I am not sure this is what we expected. This is correct. We should run global commands as we are dumping those even if we don't dump any database. Apart from these, I merged v15 delta to print db names. Either we can print the db name or

Re: Non-text mode for pg_dumpall

tricky, either more words in the doc explain the > scarenio where --file option is specified > or disallow --file option when --exclude-database is specified. We will do some more doc changes for this in next versions. > > we need to update pg_restore.sgml about MAX_ON_EXIT_NICELY 100? T

Re: Non-text mode for pg_dumpall

file has any corrupted data. */ > + if (!OidIsValid(db_oid) || strlen(dbname) == 0) > + pg_fatal("invalid entry in map.dat file at line : %d", count + 1); > + > + /* > + * XXX : before adding dbname into list, we can verify that this db > + * needs to skipped for restore or not but as of now, we are making > + * a list of all the databases. > + */ > + simple_db_oid_list_append(dbname_oid_list, db_oid, dbname); > + count++; > + } > > > db_oid first should be set to 0, dbname first character first should be set to 0 > (chardbname[0] = '\0') before sscanf call. > so if sscanf fail, the db_oid and dbname value is not undermined) Okay. Fixed. Here, I am attaching an updated patch for review and testing. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v14_pg_dumpall-with-non-text_format-3rd_feb.patch Description: Binary data

getting "shell command argument contains a newline or carriage return:" error with pg_dumpall when db name have new line in double quote

name='dbstr1; > dbstr 2'" > After this message, we are stopping the dump. I think, if we are allowing new lines in the db name, then we should dump it. Please let me know your thoughts about this error. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Non-text mode for pg_dumpall

On Fri, 24 Jan 2025 at 20:50, jian he wrote: > > On Thu, Jan 23, 2025 at 6:35 PM Mahendra Singh Thalor > wrote: > > hi. > After some tests and thinking about your reply, I admit that using > expand_dbname_patterns > in pg_restore will not work. > We need to do p

Re: Non-text mode for pg_dumpall

ch. In offline discussion, Andew already reported this test case. I will fix this in the next version. > > Regards, > Srinath Reddy Sadipiralla, > EDB: http://www.enterprisedb.com > -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Non-text mode for pg_dumpall

sed on your > v11_pg_dumpall-with-directory-tar-custom-format-21-jan.patch. I think, instead of char, if we read line by line, then we don't need that much code and need not to worry about double quotes. In the next version, I will merge some patches and will change it to read line by line. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Non-text mode for pg_dumpall

o deleted the function is_full_pattern. > > > I use > $BIN10/pg_restore --exclude-database=*x* --exclude-database=*s* > --exclude-database=*t* --verbose --file=test.sql x1.dump > the verbose message to verify my changes. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Non-text mode for pg_dumpall

ere will be lots of schema elements already exists ERROR. > but the above command case, pg_restore only restores the content/data > not schema, that's why there is very little error happening. > so here pg_restore not restore schema seems not ok? > > > 2. pg_dumpall with non-te

Re: Non-text mode for pg_dumpall

ngle db. As of now I removed this error and changed this to pg_log_info. > > > in doc/src/sgml/ref/pg_restore.sgml > > ... > --format= class="parameter">format > also need > plain > ? plain format is not supported with pg_restore. I added an error for this format. Here, I am attaching an updated patch for review and testing. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v10_pg_dumpall-with-directory-tar-custom-format-17-jan.patch Description: Binary data

Re: Non-text mode for pg_dumpall

Thanks Jian for the review and testing. On Wed, 15 Jan 2025 at 14:29, jian he wrote: > > On Sun, Jan 12, 2025 at 5:31 AM Mahendra Singh Thalor > wrote: > > > > > > > > you also need change > > > > > > -f > > class=&quo

Re: Non-text mode for pg_dumpall

Maybe create a new file for connection > management routines? (since this clearly doesn't fit common.c nor > dumputils.c). I will make a new file in follow-up patches. > On Sat, 11 Jan 2025 at 21:38, Mahendra Singh Thalor > wrote: > > > On Sat, 11 Jan 2025 at 9:30 P

Re: Non-text mode for pg_dumpall

als-only > pg_restore --exclude-database=pattern > be in a separate patch? > > > i am also wondering what will happen: > pg_restore --exclude-database=pattern --dbname=pattern For restore, we will make server connection with ‘pattern’ database and we will skip restoring for ‘pattern’ database as we are giving ‘pattern’ with —exclude-database. With server connection, we will restore global.dat at the start of pg_restore. Thanks and regards Mahendra Singh Thalor EDB postgres >

Re: Non-text mode for pg_dumpall

copied this function from pg_dumpall.c. > suggest they should be unified. Maybe create a new file for connection > management routines? (since this clearly doesn't fit common.c nor > dumputils.c). Sure. I will create a new file and I will move these common functions into that. Thanks a

Re: Non-text mode for pg_dumpall

cases. Thanks. I am also working on test cases. I will add all error test cases in the next version and will include these two also. > > you also need change > > -f class="parameter">filename > --file= class="parameter">filename > > > Send output to the specified file. If this is omitted, the > standard output is used. > > > > ? > > since if --format=d, > --file=filename > can not be omitted. Okay. I will fix it. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Non-text mode for pg_dumpall

On Thu, 9 Jan 2025 at 08:11, Mahendra Singh Thalor wrote: > > On Thu, 9 Jan 2025 at 02:30, Guillaume Lelarge wrote: > > > > Hi, > > > > Le mer. 8 janv. 2025 à 17:41, Mahendra Singh Thalor a écrit : > >> > >> On Wed, 8 Jan 2025 at 20:07, Ma

Re: Non-text mode for pg_dumpall

On Thu, 9 Jan 2025 at 02:30, Guillaume Lelarge wrote: > > Hi, > > Le mer. 8 janv. 2025 à 17:41, Mahendra Singh Thalor a > écrit : >> >> On Wed, 8 Jan 2025 at 20:07, Mahendra Singh Thalor >> wrote: >> > >> > Hi all, >> > >>

Re: Non-text mode for pg_dumpall

On Wed, 8 Jan 2025 at 20:07, Mahendra Singh Thalor wrote: > > Hi all, > > On Wed, 8 Jan 2025 at 00:34, Mahendra Singh Thalor wrote: > > > > On Mon, 6 Jan 2025 at 23:05, Nathan Bossart > > wrote: > > > > > > On Thu, Jan 02, 2025 at 02:05:13AM +05

Re: Non-text mode for pg_dumpall

Hi all, On Wed, 8 Jan 2025 at 00:34, Mahendra Singh Thalor wrote: > > On Mon, 6 Jan 2025 at 23:05, Nathan Bossart wrote: > > > > On Thu, Jan 02, 2025 at 02:05:13AM +0530, Mahendra Singh Thalor wrote: > > > Here, I am attaching an updated patch. I fixed some bugs of v

Re: Non-text mode for pg_dumpall

On Mon, 6 Jan 2025 at 23:05, Nathan Bossart wrote: > > On Thu, Jan 02, 2025 at 02:05:13AM +0530, Mahendra Singh Thalor wrote: > > Here, I am attaching an updated patch. I fixed some bugs of v01 patch and > > did some code cleanup also. > > Thank you for picking this up!

Re: Non-text mode for pg_dumpall

: exclude-database=NAME, for pg_restore, I am using NAME as of now, I will try to make it PATTERN. PATTERN should be matched from map.dat file. Please have a look over the patch and let me know feedback. On Tue, 31 Dec 2024 at 23:53, Mahendra Singh Thalor wrote: > Hi all, > With the h

Re: Non-text mode for pg_dumpall

t; about making it more convenient :) > > -- > Magnus Hagander > Me: https://www.hagander.net/ <http://www.hagander.net/> > Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/> > -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v01_poc_pg_dumpall_with_directory_31dec.patch Description: Binary data

Re: Collecting statistics about contents of JSONB columns

On Fri, 11 Mar 2022 at 04:29, Nikita Glukhov wrote: > > > On 04.02.2022 05:47, Tomas Vondra wrote: > > On 1/25/22 17:56, Mahendra Singh Thalor wrote: > > > > ... > > For the last few days, I was trying to understand these patches, and based on Tomas's su

Re: Collecting statistics about contents of JSONB columns

en keep at least frequencies for the non-analyzed paths. Next, I will take the latest patches from Nikita's last email and I will do more tests. Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Collecting statistics about contents of JSONB columns

retty 328 kB (1 row) -- *Without json patches:* postgres=# analyze test ; ANALYZE *Time: 120.864* ms postgres=# SELECT pg_size_pretty( pg_total_relation_size('pg_catalog.pg_statistic') ); pg_size_pretty 272 kB I haven't found the root cause of t

Re: Replication & recovery_min_apply_delay

Thanks Dilip and Bharath for the review. I am working on review comments and will post an updated patch. On Wed, 10 Nov 2021 at 15:31, Bharath Rupireddy wrote: > > On Wed, Oct 27, 2021 at 1:02 AM Mahendra Singh Thalor > wrote: > > On Mon, 3 Aug 2020 at 15:02, Daniel Gu

Re: Replication & recovery_min_apply_delay

ook v3[1] patch from this thread and re-based on commit head(5fedf7417b69295294b154a21). Please find the attached patch for review. link [1] : v3 patch <https://www.postgresql.org/message-id/CANXE4TeinQdw%2BM2Or0kTR24eRgWCOg479N8%3DgRvj9Ouki-tZFg%40mail.gmail.com> -- Thanks and Regards Ma

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

On Sun, 22 Aug 2021 at 22:53, Sadhuprasad Patro wrote: > > > On 2021/08/20 11:07, Mahendra Singh Thalor wrote: > > > 1) > > > Resets statistics for a single table or index in the current database > > > -to zero. > > > +

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

On Fri, 20 Aug 2021 at 07:37, Mahendra Singh Thalor wrote: > > On Fri, 20 Aug 2021 at 06:32, Sadhuprasad Patro wrote: > > > > > If we do support resetting the stats for shared tables in > > > 'pg_stat_reset', which is for DB level, > > > then th

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

* table.* table should be replaced with 'object' as we have table, index, toast for shared tables and if we can modify the above comment with some additional info, then it will be good. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

On Wed, 11 Aug 2021 at 09:17, Dilip Kumar wrote: > > On Tue, Aug 10, 2021 at 10:49 PM Mahendra Singh Thalor > wrote: > > > I checked this and found that we already have one process "stats > > collector" and in v02 patch, we are sending requests to collect sta

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

On Tue, 10 Aug 2021 at 22:32, Mahendra Singh Thalor wrote: > > On Tue, 10 Aug 2021 at 21:53, Sadhuprasad Patro wrote: > > > > > As of now, we are adding handling inside pg_stat_reset for shared > > > tables but I think we can add a new function with the name of >

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

be better if we can reset stats in pgstat_recv_resetcounter for shared tables also because shared tables are not much in counting so it will be good if we reset in one function only. I will debug this part more and will see. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: "ERROR: cache lookup failed for function %d" while dropping function from another session

On Mon, 9 Aug 2021 at 11:07, Mahendra Singh Thalor wrote: > > Hi, > I am able to hit "ERROR: cache lookup failed for function %d" while I am dropping function from another session. > > Reproducible steps are as(I tried on current head d8a75b1308b133502ae3): > >

"ERROR: cache lookup failed for function %d" while dropping function from another session

.c:1430 > #15 0x55e0b808dbe9 in main (argc=3, argv=0x55e0ba8fee70) at main.c:199 > cache lookup failed errors are never an expected behavior(Thanks Robert Hass for your opinion) so I think we should fix this error. I haven't debugged it so I will debug it and will post my findings in the coming days. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

; > > pg_stat_reset_single_table_counters > > > - > > > > > > (1 row) > > > > > > postgres=# SELECT * FROM pg_statio_all_tables where relid = > > > 'pg_database'::regclass::oid; > > &g

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

On Sat, 7 Aug 2021 at 11:49, Mahendra Singh Thalor wrote: > > On Sat, 7 Aug 2021 at 00:13, Mahendra Singh Thalor wrote: > > > > On Fri, 6 Aug 2021 at 21:17, Dilip Kumar wrote: > > > > > > On Fri, Aug 6, 2021 at 8:53 PM Himanshu Upadhyaya &g

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

On Sat, 7 Aug 2021 at 00:13, Mahendra Singh Thalor wrote: > > On Fri, 6 Aug 2021 at 21:17, Dilip Kumar wrote: > > > > On Fri, Aug 6, 2021 at 8:53 PM Himanshu Upadhyaya > > wrote: > > > > > > Hi Sadhu, > > > > > > Patch working as expe

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

e | relname | heap_blks_read | heap_blks_hit | idx_blks_read | idx_blks_hit | toast_blks_read | toast_blks_hit | tidx_blks_read | tidx_blks_hit ---++-++---+---+--+-+++--- 1262 | pg_catalog | pg_database | 0 | 0 | 2 |0 | 0 | 0 | 0 | 0 (1 row) postgres=# -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com v0002-pg_stat_reset-and-pg_stat_reset_single_table_counter.patch Description: Binary data

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

d -- SharedDependRelationId 1214 7. pg_shseclabel -- SharedSecLabelRelationId 3592 8. pg_db_role_setting -- DbRoleSettingRelationId 2694 9. pg_replication_origin -- ReplicationOriginRelationId 6000 10. pg_subscription -- SubscriptionRelationId 6100 -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://w

Re: Support reset of Shared objects statistics in "pg_stat_reset" function

ing to do if not there. */ dbentry = pgstat_get_db_entry(msg->m_databaseid, false); - if (!dbentry) return; I think, by mistake, you removed one line in the patch. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Fix uninitialized variable access (src/backend/utils/mmgr/freepage.c)

> if (result.index > 0) /* result.index is garbage or invalid here) */ > > regards, > Ranier Vilela -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Can we remove extra memset in BloomInitPage, GinInitPage and SpGistInitPage when we have it in PageInit?

On Tue, 6 Apr 2021 at 19:14, Bharath Rupireddy wrote: > > On Tue, Apr 6, 2021 at 6:09 PM Michael Paquier wrote: > > > > On Mon, Mar 22, 2021 at 10:58:17AM +0530, Mahendra Singh Thalor wrote: > > > Your changes look to fine me and I am also not getting any failure. I

Re: UPDATE ... SET (single_column) = row_constructor is a bit broken from V10 906bfcad7ba7c

On Tue, 23 Mar 2021 at 02:43, Justin Pryzby wrote: > > On Mon, Mar 22, 2021 at 02:10:49PM +0530, Mahendra Singh Thalor wrote: > > Hi Hackers, > > > > Commit 906bfcad7ba7c has improved handling for "UPDATE ... SET > > (column_list) = row_constructor",

UPDATE ... SET (single_column) = row_constructor is a bit broken from V10 906bfcad7ba7c

e above test case is passing in all the branches. This looks like a bug. Thoughts? Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Can we remove extra memset in BloomInitPage, GinInitPage and SpGistInitPage when we have it in PageInit?

ev system with the > attached patch. > > > Thoughts? Your changes look to fine me and I am also not getting any failure. I think we should back-patch all the branches. Patch is applying to all the branches(till v95) and there is no failure. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

usage in > README.tuplock would definitely be useful, especially since the combination > isn't always produced. How about adding something like: > > HEAP_KEYS_UPDATED > This bit lives in t_infomask2. If set, indicates that the XMAX updated > this tuple and changed the key values, or it deleted the tuple. > + It can also be set in combination of HEAP_XMAX_LOCK_ONLY. > It's set regardless of whether the XMAX is a TransactionId or a MultiXactId. Make sense. Please can you update this? -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

in exec_simple_query (query_string=0x5612a787b570 > "UPDATE t1 SET val = 'hoho' WHERE id = 2;") at postgres.c:1240 > #18 0x5612a612b8dd in PostgresMain (argc=1, argv=0x7ffc8b5e3790, > dbname=0x5612a78a74f0 "postgres", username=0x5612a78a74c8 "

Re: display offset along with block number in vacuum errors

mation, so I think we should back-patch to 13. The > second one is to add additional vacuum error context information, so > that is for only HEAD. Does that make sense? Also, let me know if you > have any more comments. Thanks Amit for updating the patch. All changes in v7-02 look fine to me. Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com > > -- > With Regards, > Amit Kapila. -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

Re: display offset along with block number in vacuum errors

Thanks Sawada and Justin. On Sun, 2 Aug 2020 at 09:33, Masahiko Sawada wrote: > > On Sat, 1 Aug 2020 at 16:02, Mahendra Singh Thalor wrote: > > > > Thanks Justin. > > > > On Sat, 1 Aug 2020 at 11:47, Justin Pryzby wrote: > > > > > > On Fri, Jul 3

Re: display offset along with block number in vacuum errors

t; whoops > > > On Wed, Jul 29, 2020 at 12:35:17AM +0530, Mahendra Singh Thalor wrote: > > > > Here: > > > > > > > > @@ -1924,14 +1932,22 @@ lazy_vacuum_page(Relation onerel, BlockNumber > > > > blkno, Buffer buffer, > > > &

Re: display offset along with block number in vacuum errors

Thanks Justin, Sawada and Michael for reviewing. On Mon, 27 Jul 2020 at 16:43, Justin Pryzby wrote: > > On Fri, Jul 24, 2020 at 11:18:43PM +0530, Mahendra Singh Thalor wrote: > > Hi hackers, > > We discussed in another email thread[1], that it will be helpful if we can > &

Re: display offset along with block number in vacuum errors

Thanks Michael for looking into this. On Sat, 25 Jul 2020 at 15:02, Michael Paquier wrote: > > On Fri, Jul 24, 2020 at 11:18:43PM +0530, Mahendra Singh Thalor wrote: > > In commit b61d161(Introduce vacuum errcontext to display additional > > information), we added vacuum err

display offset along with block number in vacuum errors

errmsg_internal("for block %u and offnum %u, found xmin %u from before relfrozenxid %u", + ItemPointerGetBlockNumber(tid), + ItemPointerGetOffsetNumber(tid), xid, relfrozenxid))); Attaching v01_0002 patch for this method. Please let me know your thoughts. [1] : http://postgr.es/m/

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Fri, 29 May 2020 at 15:52, Amit Kapila wrote: > > On Wed, May 27, 2020 at 5:19 PM Mahendra Singh Thalor wrote: >> >> On Tue, 26 May 2020 at 16:46, Amit Kapila wrote: >> >> Hi all, >> On the top of v16 patch set [1], I did some testing for DDL's a

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

ix (DDL & DML), it is 2-10%. why are we seeing 11-13 % of the extra wall, basically, the amount of extra WAL is not very high but the amount of WAL generated with add column int/date is just ~1000 bytes so additional 100 bytes will be around 10% and for add column text it is ~35000 bytes so % is less. For text, these ~35000 bytes are due to toast. [1]: https://www.postgresql.org/message-id/CAFiTN-vnnrk580ucZVYnub_UQ-ayROew8fQ2Yn5aFYMeF0U03w%40mail.gmail.com [2]: https://docs.google.com/spreadsheets/d/1g11MrSd_I39505OnGoLFVslz3ykbZ1nmfR_gUiE_O9k/edit?usp=sharing -- Thanks and Regards Mahendra Singh Thalor EnterpriseDB: http://www.enterprisedb.com

  1   2   >