Re: csv format for psql

2018-11-26 Thread Daniel Verite
Tom Lane wrote: > OK, reasonable arguments were made why not to allow multi-character > separators. Should we then match the server and insist on a single-byte > separator? It's a bit inconsistent if psql can be made to emit "csv" > files that COPY can't read, especially when it's otherw

Re: csv format for psql

2018-11-26 Thread Daniel Verite
I wrote: > A proposed fix is attached. print_csv_vertical() is left unchanged > because it's not possible currently to end up with \. alone > on a line with the expanded display On second thought, it is possible postgres=# \pset format csv Output format is csv. postgres=# \pset fieldsep

Re: csv format for psql

2018-11-27 Thread Daniel Verite
Tom Lane wrote: > what I did instead was just to make > csv_print_field force field quoting if any of these cases could > possibly apply. That will result in excess quoting in some > cases, but I think that's fine, since they're all pretty uncommon. Indeed. > (BTW, it seems only chance

Re: psql --csv and other parameters

2018-11-28 Thread Daniel Verite
Erik Rijkers wrote: > I don't know if this really is a bug but it seems wrong to me: > > psql -A --csv -Xc "select * from pg_namespace order by 1" > > gives a difference result than > > psql --csv -A -Xc "select * from pg_namespace order by 1" -A and --csv each select an output format,

Re: Markdown format output for psql, design notes

2018-11-28 Thread Daniel Verite
Lætitia Avrot wrote: > # The result I want > From points 3 and 4, here is what I'd like to see : > > | Header 1 | Header 2 | Header 3 | > |--|--|--| > | content | content | content | > | content | content | content | > (2 rows) What would it look like when a field

Re: Markdown format output for psql, design notes

2018-11-29 Thread Daniel Verite
Lætitia Avrot wrote: > I suppose you mean in the standard output when the screen is too short to > print the whole line ? > Because if the output is redirected to a file (with `\o myfile` for > example), the line end naturally when the row ends. No I meant independently of the screen, if

Re: NEXT VALUE FOR sequence

2018-02-19 Thread Daniel Verite
Laurenz Albe wrote: > The SQL standard has the expression "NEXT VALUE FOR asequence" to do > what we traditionally do with "nextval('asequence')". The behavior mandated by the standard is that several invocations of NEXT VALUE on the same sequence on the same output row must produce the s

Re: csv format for psql

2018-03-01 Thread Daniel Verite
Fabien COELHO wrote: > Maybe some \csv command could set the format to csv, fieldsep to ",", > tuples_only to on, recordsep to '\n'? Not sure whether it would be > acceptable, though, and how to turn it off once turned on... Probably an > average (aka not good) idea:-) Thanks for revie

Re: pgbench - add \if support

2018-03-01 Thread Daniel Verite
Fabien COELHO wrote: > There does not seem to be any on my side: > > sh> hexdump pgbench-if-6.patch | grep '0[ad]' | head -3 > 040 6562 636e 2e68 6773 6c6d 690a 646e 7865 > 060 2031 3031 3630 3434 2d0a 2d2d 6120 642f > 080 6770 6562 636e 2e68 6773 6c6d 2b0a 2b2b

Re: csv format for psql

2018-03-07 Thread Daniel Verite
I wrote: > a better idea would to have a new \pset fieldsep_csv PFA a v3 patch that implements that, along with regression tests this time. Best regards, -- Daniel Vérité PostgreSQL-powered mailer: http://www.manitou-mail.org Twitter: @DanielVerite diff --git a/doc/src/sgml/ref/psql-ref.sgml b

Re: csv format for psql

2018-03-07 Thread Daniel Verite
David Fetter wrote: > This seems pretty specialized. If we're adding something new, how about > >psql --format=csv -o foo.csv -c 'TABLE foo' It's a bit easier to memorize than -P format=csv, but psql doesn't have any long option that does not a have a short form with a single lette

Re: csv format for psql

2018-03-07 Thread Daniel Verite
David Fetter wrote: > We have some inconsistency here in that fewer table formats are > supported, but I think asciidoc, etc., do this correctly via > invocations like: > >psql -P format=asciidoc -o foo.adoc -AtXc 'TABLE foo' -A is equivalent to -P format=unaligned, so in the above i

Re: csv format for psql

2018-03-09 Thread Daniel Verite
Fabien COELHO wrote: > I also think that a short option brings little value, and "--csv" is good > enough for the purpose, so I would agree to remove the "-C" binding. It's not that accepting -C brings much value by itself, it's that loosing the consistency across all options comes with

Re: csv format for psql

2018-03-09 Thread Daniel Verite
I wrote: > recordsep in the unaligned mode doesn't play the role of a line ending > because the last line is not finished by recordsep. According to the source >code, this is intended, see print_unaligned_text() in print.c: Something else comes to mind: CSV allows linefeeds inside fields,

Re: csv format for psql

2018-03-09 Thread Daniel Verite
David G. Johnston wrote: > I'm not following - if recordsep is not something that would > interpreted as a newline then the file we output would have not > structural newlines. > It might have data newlines but those would be quoted. They would be, but I don't quite understand the point i

Re: Built-in CTYPE provider

2024-01-10 Thread Daniel Verite
Jeff Davis wrote: > Attached a more complete version that fixes a few bugs [v15 patch] When selecting the builtin provider with initdb, I'm getting the following setup: $ bin/initdb --locale=C.UTF-8 --locale-provider=builtin -D/tmp/pgdata The database cluster will be initialized wit

Re: Built-in CTYPE provider

2024-01-12 Thread Daniel Verite
Jeff Davis wrote: > > Jeremy also raised a problem with old versions of psql connecting to > > a > > new server: the \l and \dO won't work. Not sure exactly what to do > > there, but I could work around it by adding a new field rather than > > renaming (though that's not ideal). > > I did

Re: Built-in CTYPE provider

2024-01-15 Thread Daniel Verite
Jeff Davis wrote: > New version attached. [v16] Concerning the target category_test, it produces failures with versions of ICU with Unicode < 15. The first one I see with Ubuntu 22.04 (ICU 70.1) is: category_test: Postgres Unicode version:15.1 category_test: ICU Unicode version:

Re: Fixing backslash dot for COPY FROM...CSV

2024-01-16 Thread Daniel Verite
Robert Haas wrote: > Part of my hesitancy, I suppose, is that I don't > understand why we even have this strange convention of making \. > terminate the input in the first place -- I mean, why wouldn't that be > done in some kind of out-of-band way, rather than including a special > marker

Re: Built-in CTYPE provider

2024-01-18 Thread Daniel Verite
Peter Eisentraut wrote: > > If the Postgres default was bytewise sorting+locale-agnostic > > ctype functions directly derived from Unicode data files, > > as opposed to libc/$LANG at initdb time, the main > > annoyance would be that "ORDER BY textcol" would no > > longer be the human-favor

Re: Fixing backslash dot for COPY FROM...CSV

2024-01-24 Thread Daniel Verite
Robert Haas wrote: > Those links unfortunately seem not to be entirely specific to this > issue. Other, related things seem to be discussed there, and it's not > obvious that everyone agrees on what to do, or really that anyone > agrees on what to do. The best link that I found for this ex

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-01-30 Thread Daniel Verite
vignesh C wrote: > patching file src/interfaces/libpq/exports.txt > Hunk #1 FAILED at 191. > 1 out of 1 hunk FAILED -- saving rejects to file > src/interfaces/libpq/exports.txt.rej > > Please post an updated version for the same. PFA a rebased version. Best regards, -- Daniel Vérité h

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-03-01 Thread Daniel Verite
I wrote: > Here's a POC patch implementing row-by-row fetching. PFA an updated patch. Best regards, -- Daniel Vérité https://postgresql.verite.pro/ Twitter: @DanielVerite diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c index f907f5d4e8..ad5e8a5de9 100644 --- a/src/bin/psql/common.

Re: Built-in CTYPE provider

2024-03-27 Thread Daniel Verite
Jeff Davis wrote: > The tests include initcap('123abc') which is '123abc' in the PG_C_UTF8 > collation vs '123Abc' in PG_UNICODE_FAST. > > The reason for the latter behavior is that the Unicode Default Case > Conversion algorithm for toTitlecase() advances to the next Cased > character be

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-01 Thread Daniel Verite
Laurenz Albe wrote: > I had a look at patch 0001 (0002 will follow). Thanks for reviewing this! I've implemented the suggested doc changes. A patch update will follow with the next part of the review. > > --- a/src/interfaces/libpq/fe-exec.c > > +++ b/src/interfaces/libpq/fe-exec.c > >

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-01 Thread Daniel Verite
Laurenz Albe wrote: > Here is the code review for patch number 2: > +static void > +CloseGOutput(FILE *gfile_fout, bool is_pipe) > > It makes sense to factor out this code. > But shouldn't these functions have a prototype at the beginning of the file? Looking at the other static functio

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-02 Thread Daniel Verite
Tom Lane wrote: > I do not buy that psql's FETCH_COUNT mode is a sufficient reason > to add it. FETCH_COUNT mode is not something you'd use > non-interactively I should say that I've noticed significant latency improvements with FETCH_COUNT retrieving large resultsets, such that it would

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-02 Thread Daniel Verite
Tom Lane wrote: > > I should say that I've noticed significant latency improvements with > > FETCH_COUNT retrieving large resultsets, such that it would benefit > > non-interactive use cases. > > Do you have a theory for why that is? It's pretty counterintuitive > that it would help at a

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-05 Thread Daniel Verite
Tom Lane wrote: > I've looked over this patch and I generally agree that this is a > reasonable solution. Thanks for reviewing this! > I'm also wondering why the patch adds a test for > "PQprotocolVersion(conn) >= 3" in handleCopyIn. I've removed this in the attached update. > I concu

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-05 Thread Daniel Verite
Tom Lane wrote: > Not sure what to do here. One idea is to install just the psql-side > fix, which should break nothing now that version-2 protocol is dead, > and then wait a few years before introducing the server-side change. > That seems kind of sad though. Wouldn't backpatching solve

Re: Fixing backslash dot for COPY FROM...CSV

2024-04-06 Thread Daniel Verite
Tom Lane wrote: > This is sufficiently weird that I'm starting to come around to > Daniel's original proposal that we just drop the server's recognition > of \. altogether (which would allow removal of some dozens of lines of > complicated and now known-buggy code) FWIW my plan was to not

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Daniel Verite
Tom Lane wrote: > I've reconsidered after realizing that implementing FETCH_COUNT > atop traditional single-row mode would require either merging > single-row results into a bigger PGresult or persuading psql's > results-printing code to accept an array of PGresults not just > one. Either

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-04-08 Thread Daniel Verite
Alexander Lakhin wrote: > >> Now that ExecQueryUsingCursor() is gone, it's not clear, what does > >> the following comment mean:? > >> * We must turn off gexec_flag to avoid infinite recursion. Note that > >> * this allows ExecQueryUsingCursor to be applied to the individual >

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-02-12 Thread Daniel Verite
Jakub Wartak wrote: > when I run with default pager (more or less): > \set FETCH_COUNT 1000 > WITH data AS (SELECT generate_series(1, 2000) as Total) select > repeat('a',100) || data.Total || repeat('b', 800) as total_pat from > data; > -- it enters pager, a skip couple of pages and t

TAP tests for psql \g piped into program

2023-01-02 Thread Daniel Verite
Hi, This is a follow-up to commit d2a44904 from the 2022-11 CF [1] The TAP tests were left out with the suggestion to use Perl instead of cat (Unix) / findstr (Windows) as the program to pipe into. PFA a patch implementing that suggestion. [1] https://commitfest.postgresql.org/40/4000/ Best re

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-04 Thread Daniel Verite
Jakub Wartak wrote: > It might be a not so well known fact (?) that CTEs are not executed > with cursor when asked to do so, but instead silently executed with > potential huge memory allocation going on. Patch is attached. My one > doubt is that not every statement starting with "WITH" is

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-01-12 Thread Daniel Verite
Tom Lane wrote: > I agree that it seems like a good idea to try. > There will be more per-row overhead, but the increase in flexibility > is likely to justify that. Here's a POC patch implementing row-by-row fetching. If it wasn't for the per-row overhead, we could probably get rid of Ex

Re: TAP tests for psql \g piped into program

2023-03-29 Thread Daniel Verite
Peter Eisentraut wrote: > So for your patch, I would just do the path adjustment ad hoc in-line. > It's just one additional line. Here's the patch updated that way. Best regards, -- Daniel Vérité https://postgresql.verite.pro/ Twitter: @DanielVerite diff --git a/src/bin/psql/t/001_bas

pg_collation.collversion for C.UTF-8

2023-04-18 Thread Daniel Verite
Hi, get_collation_actual_version() in pg_locale.c currently excludes C.UTF-8 (and more generally C.*) from versioning, which makes pg_collation.collversion being empty for these collations. char * get_collation_actual_version(char collprovider, const char *collcollate) { if (collpr

Re: pg_collation.collversion for C.UTF-8

2023-04-22 Thread Daniel Verite
Thomas Munro wrote: > It looks like for technical reasons > inside glibc, that couldn't be done before 2.35: > > https://sourceware.org/bugzilla/show_bug.cgi?id=17318 > > That strengthens my opinion that C.UTF-8 (the real C.UTF-8 supplied > by the glibc project) isn't supposed to be vers

Re: Order changes in PG16 since ICU introduction

2023-04-25 Thread Daniel Verite
Jeff Davis wrote: > > (I'm not sure whether those operations can get redirected to ICU > > today > > or whether they still always go to libc, but we'll surely want to fix > > it eventually if the latter is still true.) > > Those operations do get redirected to ICU today. FTR the full te

Re: Add standard collation UNICODE

2023-04-27 Thread Daniel Verite
Peter Eisentraut wrote: > COLLATE UNICODE > > instead of > > COLLATE "und-x-icu" > > or whatever it is, is pretty useful. > > So, attached is a small patch to add this. This collation has an empty pg_collation.collversion column, instead of being set to the same value as "un

Re: Order changes in PG16 since ICU introduction

2023-04-27 Thread Daniel Verite
Jeff Davis wrote: > Attached are a few small patches: > > 0001: don't convert C to en-US-u-va-posix > 0002: handle locale C the same regardless of the provider, as you > suggest above > 0003: make LOCALE (or --locale) apply to everything including ICU Testing this briefly I noticed

Re: Add header support to text format and matching feature

2022-03-25 Thread Daniel Verite
Peter Eisentraut wrote: > - The DefGetCopyHeader() function seems very bulky and might not be > necessary. I think you can just check for the string "match" first and > then use defGetBoolean() as before if it didn't match. The problem is that defGetBoolean() ends like this in the non-

Re: variable filename for psql \copy

2022-04-25 Thread Daniel Verite
Jiří Fejfar wrote: > I have found maybe buggy behaviour (of psql parser?) when using psql \copy > with psql variable used for filename. While it's annoying that it doesn't work as you tried it, this behavior is documented, so in that sense it's not a bug. The doc also suggests a workarou

Re: Pre-proposal: unicode normalized text

2023-10-17 Thread Daniel Verite
Jeff Davis wrote: > I believe the patch has utility as-is, but I've been brainstorming a > few more ideas that could build on it: > > * Add a per-database option to enforce only storing assigned unicode > code points. There's a problem in the fact that the set of assigned code points is

Re: Does UCS_BASIC have the right CTYPE?

2023-10-26 Thread Daniel Verite
Peter Eisentraut wrote: > > That seems to suggest the standard answer should be 'Á' regardless of > > any COLLATE clause (though I could be misreading). I'm a bit confused > > by that... what's the standard-compatible way to specify the locale for > > UPPER()/LOWER()? If there is none, the

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2023-11-20 Thread Daniel Verite
Hi, Here's a new version to improve the performance of FETCH_COUNT and extend the cases when it can be used. Patch 0001 adds a new mode in libpq to allow the app to retrieve larger chunks of results than the single row of the row-by-row mode. The maximum number of rows per PGresult is set by the

Re: [ psql - review request ] review request for \d+ tablename, \d+ indexname indenting

2023-11-22 Thread Daniel Verite
Shlok Kyal wrote: > > The error was corrected and a new diff file was created. > > The diff file was created based on 16 RC1. > > We confirmed that 5 places where errors occurred when performing > > make check were changed to ok. Reviewing the patch, I see these two problems in the curren

Re: proposal: change behavior on collation version mismatch

2023-11-28 Thread Daniel Verite
Jeremy Schneider wrote: > 1) "collation changes are uncommon" (which is relatively correct) > 2) "most users would rather have ease-of-use than 100% safety, since > it's uncommon" > > And I think this led to the current behavior of issuing a warning rather > than an error, There's a tech

Re: EXCLUDE COLLATE in CREATE/ALTER TABLE document

2023-12-01 Thread Daniel Verite
shihao zhong wrote: > Thanks for your comments, a new version is attached. In this hunk: @@ -1097,8 +1097,8 @@ WITH ( MODULUS numeric_literal, REM method index_method. The operators are required to be commutative. Each exclude_element - can optionally specify an

Re: Make COPY format extendable: Extract COPY TO format implementations

2023-12-06 Thread Daniel Verite
Sutou Kouhei wrote: > * 2022-04: Apache Arrow [2] > * 2018-02: Apache Avro, Apache Parquet and Apache ORC [3] > > (FYI: I want to add support for Apache Arrow.) > > There were discussions how to add support for more formats. [3][4] > In these discussions, we got a consensus about making

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Daniel Verite
Andrew Dunstan wrote: > IMNSHO, we should produce either a single JSON > document (the ARRAY case) or a series of JSON documents, one per row > (the LINES case). "COPY Operations" in the doc says: " The backend sends a CopyOutResponse message to the frontend, followed by zero or mor

Re: Emitting JSON to file using COPY TO

2023-12-07 Thread Daniel Verite
Joe Conway wrote: > The attached should fix the CopyOut response to say one column. I.e. it > ought to look something like: Spending more time with the doc I came to the opinion that in this bit of the protocol, in CopyOutResponse (B) ... Int16 The number of columns in the data to be cop

Re: Emitting JSON to file using COPY TO

2023-12-08 Thread Daniel Verite
Joe Conway wrote: > copyto_json.007.diff When the source has json fields with non-significant line feeds, the COPY output has these line feeds too, which makes the output incompatible with rule #2 at https://jsonlines.org ("2. Each Line is a Valid JSON Value"). create table j(f json);

Re: Emitting JSON to file using COPY TO

2023-12-08 Thread Daniel Verite
Dave Cramer wrote: > > This argument for leaving 3 as the column count makes sense to me. I > > agree this content is not meant to facilitate interpreting the contents at > > a protocol level. > > > > I'd disagree. From my POV if the data comes back as a JSON Array this is > one object a

Re: Built-in CTYPE provider

2023-12-13 Thread Daniel Verite
Jeff Davis wrote: > While "full" case mapping sounds more complex, there are actually > very few cases to consider and they are covered in another (small) > data file. That data file covers ~100 code points that convert to > multiple code points when the case changes (e.g. "ß" -> "SS"), 7

Fixing backslash dot for COPY FROM...CSV

2023-12-18 Thread Daniel Verite
Hi, PFA a patch that attempts to fix the bug that \. on a line by itself is handled incorrectly by COPY FROM ... CSV. This issue has been discussed several times previously, for instance in [1] and [2], and mentioned in the doc for \copy in commit 42d3125. There's one case that works today: whe

Re: Fixing backslash dot for COPY FROM...CSV

2023-12-19 Thread Daniel Verite
vignesh C wrote: > I noticed that these tests are passing without applying patch too: > +insert into copytest2(test) values('line1'), ('\.'), ('line2'); > +copy (select test from copytest2 order by test collate "C") to :'filename' > csv; > +-- get the data back in with copy > +truncate co

Re: Built-in CTYPE provider

2023-12-20 Thread Daniel Verite
Jeff Davis wrote: > But there are a lot of users for whom neither of those things are true, > and it makes zero sense to order all of the text indexes in the > database according to any one particular locale. I think these users > would prioritize stability and performance for the databas

Re: Fixing backslash dot for COPY FROM...CSV

2023-12-21 Thread Daniel Verite
vignesh C wrote: > Thanks for the updated patch, any reason why this is handled only in csv. > postgres=# copy test1 from '/home/vignesh/postgres/inst/bin/copy1.out'; > COPY 1 > postgres=# select * from test1; > c1 > --- > line1 > (1 row) I believe it's safer to not change anything t

Re: Built-in CTYPE provider

2023-12-22 Thread Daniel Verite
Robert Haas wrote: > For someone who is currently defaulting to es_ES.utf8 or fr_FR.utf8, > a change to C.utf8 would be a much bigger problem, I would > think. Their alphabet isn't in code point order, and so things would > be alphabetized wrongly. > That might be OK if they don't care ab

Re: Fixing backslash dot for COPY FROM...CSV

2023-12-31 Thread Daniel Verite
Hi, The CI patch tester fails on this patch, because it has a label at the end of a C block, which I'm learning is a C23 feature that happens to be supported by gcc 11 [1], but is not portable. PFA an update fixing this, plus removing an obsolete chunk in the COPY documentation that v2 left out

Re: psql's FETCH_COUNT (cursor) is not being respected for CTEs

2024-01-02 Thread Daniel Verite
Hi, PFA a rebased version. Best regards, -- Daniel Vérité https://postgresql.verite.pro/ Twitter: @DanielVerite From cd0fe1d517a0e31e031fbbea1e603a715c77ea97 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20V=C3=A9rit=C3=A9?= Date: Tue, 2 Jan 2024 14:15:48 +0100 Subject: [PATCH v5 1/2] Imple

Re: EBCDIC sorting as a use case for ICU rules

2023-08-30 Thread Daniel Verite
Peter Eisentraut wrote: > Committed with some editing. I moved the existing rules example from > the CREATE COLLATION page into the new section you created, so we have a > simple example followed by the complex example. OK, thanks for pushing this! Best regards, -- Daniel Vérité htt

Re: Unicode normalization SQL functions

2020-03-23 Thread Daniel Verite
Peter Eisentraut wrote: > What is that status of this patch set? I think we have nailed down the > behavior, but there were some concerns about certain performance > characteristics. Do people feel that those are required to be addressed > in this cycle? Not finding any other issue w

Re: Making psql error out on output failures

2020-03-23 Thread Daniel Verite
Peter Eisentraut wrote: > > If there's no more review to do, would you consider moving it to > > Ready for Committer? > > committed Thanks! Best regards, -- Daniel Vérité PostgreSQL-powered mailer: http://www.manitou-mail.org Twitter: @DanielVerite

Re: A bug when use get_bit() function for a long bytea string

2020-03-27 Thread Daniel Verite
Ashutosh Bapat wrote: > I think we need a similar change in byteaGetBit() and byteaSetBit() > as well. get_bit() and set_bit() as SQL functions take an int4 as the "offset" argument representing the bit number, meaning that the maximum value that can be passed is 2^31-1. But the maximum t

Re: proposal \gcsv

2020-03-28 Thread Daniel Verite
Erik Rijkers wrote: > 2. let the psql command-line option '--csv' honour the value given by > psql -F/--field-separator (it does not do so now) > > or > > 3. add an psql -commandline option: > --csv-field-separator Setting the field separator on the command line is already supporte

Re: proposal \gcsv

2020-04-01 Thread Daniel Verite
Tom Lane wrote: > I could see having a command to copy the current primary formatting > parameters to the alternate area, too. We could have a stack to store parameters before temporary changes, for instance if you want to do one csv export and come back to normal without assuming what "

Re: proposal \gcsv

2020-04-02 Thread Daniel Verite
Alvaro Herrera wrote: > Can we fix that by adding some syntax to allow command aliases? > So you could add to your .psqlrc something like > > \alias \gcsv \pset push all \; \cbuf; \; \pset pop > > where the \cbuf is a hypothetical "function" that expands to the current > query buffer. T

Re: A bug when use get_bit() function for a long bytea string

2020-04-02 Thread Daniel Verite
movead...@highgo.ca wrote: > A little update for the patch, and patches for all stable avilable. Some comments about the set_bit/get_bit parts. I'm reading long_bytea_string_bug_fix_ver6_pg10.patch, but that applies probably to the other files meant for the existing releases (I think you

Re: A bug when use get_bit() function for a long bytea string

2020-04-03 Thread Daniel Verite
movead...@highgo.ca wrote: > >I believe the formula for the upper limit in the 32-bit case is: > > (len <= PG_INT32_MAX / 8) ? (len*8 - 1) : PG_INT32_MAX; > > >These functions could benefit from a comment mentioning that > >they cannot reach the full extent of a bytea, because of the

Re: psql: Add command to use extended query protocol

2022-11-08 Thread Daniel Verite
David G. Johnston wrote: > I would keep the \gp meta-command to force extended mode regardless > of whether the query itself requires it. +1 > A pset variable to control the default seems reasonable as well. > The implication would be that if you set that pset variable there is > no way

Re: psql: Add command to use extended query protocol

2022-11-09 Thread Daniel Verite
Peter Eisentraut wrote: > Is there a use case for a global setting? I assume that we may sometimes want to use the extended protocol on all queries of a script, like pgbench does with --protocol=extended. Outside of psql, it's too complicated to parse a SQL script to replace the end-of-qu

Re: psql: Add command to use extended query protocol

2022-11-14 Thread Daniel Verite
Peter Eisentraut wrote: > > I assume that we may sometimes want to use the > > extended protocol on all queries of a script, like > > pgbench does with --protocol=extended. > > But is there an actual use case for this in psql? In pgbench, there are > scenarios where you want to test asp

Re: Tests for psql \g and \o

2022-11-23 Thread Daniel Verite
Michael Paquier wrote: > +psql_like($node, "SELECT 'one' \\g | cat >$g_file", qr//, "one command > \\g"); > +my $c1 = slurp_file($g_file); > +like($c1, qr/one/); > > Windows may not have an equivalent for "cat", no? Note that psql's > 001_basic.pl has no restriction in place for Windows.

Re: Tests for psql \g and \o

2022-11-30 Thread Daniel Verite
Michael Paquier wrote: > Thanks, the tests part of the main regression test suite look good to > me, so I have applied them after fixing a few typos and tweaking the > style of the test. Thanks! > Regarding the tests with pipes, I had cold feet with the > dependencies on cat for non-WIN3

Re: COPY table_name (single_column) FROM 'unknown.txt' DELIMITER E'\n'

2021-05-09 Thread Daniel Verite
Darafei "Komяpa" Praliaskouski wrote: > What I would prefer is some new COPY mode like RAW that will just push > whatever it gets on the stdin/input into the cell on the server side. This > way it can be proxied by psql, utilize existing infra for passing streams > and be used in shell scr

Re: Allow tailoring of ICU locales with custom rules

2023-02-04 Thread Daniel Verite
Laurenz Albe wrote: > Cool so far. Now I created a database with that locale: > > CREATE DATABASE teutsch LOCALE_PROVIDER icu ICU_LOCALE german_phone > LOCALE "de_AT.utf8" TEMPLATE template0; > > Now the rules are not in "pg_database": The parameter after ICU_LOCALE is passed dire

Re: proposal: psql: psql variable BACKEND_PID

2023-02-06 Thread Daniel Verite
Corey Huinker wrote: > Manually testing confirms that it works, at least for the connected state. I > don't actually know how get psql to invoke DISCONNECT, so I killed the dev > server and can confirm Maybe something like this could be used, with no external action: postgres=# \echo :

Re: proposal: psql: psql variable BACKEND_PID

2023-02-06 Thread Daniel Verite
I wrote: > > In the varlistentry, I suggest we add "This variable is unset when the > > connection is lost." after "but can be changed or unset. > > Personally I'd much rather have BACKEND_PID set to 0 rather than being unset > when not connected. For one thing it allows safely using \if

Re: Allow tailoring of ICU locales with custom rules

2023-02-20 Thread Daniel Verite
Peter Eisentraut wrote: [patch v5] Two quick comments: - pg_dump support need to be added for CREATE COLLATION / DATABASE - there doesn't seem to be a way to add rules to template1. If someone wants to have icu rules and initial contents to their new databases, I think they need to crea

[patch] \g with multiple result sets and \watch with copy queries

2022-09-29 Thread Daniel Verite
Hi, The psql improvement in v15 to output multiple result sets does not behave as one might expect with \g: the output file or program to pipe into is opened/closed on each result set, overwriting the previous ones in the case of \g file. Example: psql -At

Re: [patch] \g with multiple result sets and \watch with copy queries

2022-10-04 Thread Daniel Verite
Tom Lane wrote: > Pushed after making some corrections. Thanks! > Given the time pressure, I did not worry about installing regression > test coverage for this stuff, but I wonder if we shouldn't add some. Currently, test/regress/sql/psql.sql doesn't AFAICS write anything outside of std

Re: [patch] \g with multiple result sets and \watch with copy queries

2022-10-07 Thread Daniel Verite
Tom Lane wrote: > > Currently, test/regress/sql/psql.sql doesn't AFAICS write anything > > outside of stdout, but \g, \o, \copy need to write to external > > files to be tested properly. > > Yeah, I don't think we can usefully test these in psql.sql, because > file-system side effects ar

Tests for psql \g and \o

2022-11-01 Thread Daniel Verite
Hi, Here's a patch adding regression tests for \g and \o, and TAP tests for \g | program, It's a follow up to the discussion at [1]. Since this discussion already has a slot in the CF [2] with a committed patch, let's start a new separate thread. [1] https://www.postgresql.org/message-id/4333844

Re: [patch] \g with multiple result sets and \watch with copy queries

2022-11-01 Thread Daniel Verite
Corey Huinker wrote: > I think that because it is more complicated than usual psql, we may want to > comment on the intention of the tests and some of the less-than-common psql > elements (\set concatenation, resetting \o, etc). If you see value in that > I can amend the patch. If the int

Re: psql: Add command to use extended query protocol

2022-11-02 Thread Daniel Verite
Jehan-Guillaume de Rorthais wrote: > As I wrote in my TCE review, would it be possible to use psql vars to set > some > named parameters for the prepared query? This would looks like: > > \set p1 foo > \set p2 bar > SELECT :'p1', :'p2' \gp As I understand the feature, variables would

Re: ICU_LOCALE set database default icu collation but not working as intended.

2022-05-26 Thread Daniel Verite
jian he wrote: > CREATE > DATABASE dbicu1 LOCALE_PROVIDER icu LOCALE 'en_US.UTF-8' ICU_LOCALE > 'en-u-kf-upper' TEMPLATE 'template0'; > CREATE DATABASE dbicu2 LOCALE_PROVIDER icu LOCALE 'en_US.UTF-8' ICU_LOCALE > 'en-u-kr-latn-digit' TEMPLATE 'template0'; > [...] > I am not sure this is my

Re: ICU_LOCALE set database default icu collation but not working as intended.

2022-05-28 Thread Daniel Verite
jian he wrote: > - dbicu3, ICU_LOCALE 'en-u-kr-latn-digit-kf-upper-kn-true' seems > 'kf-upper' not grouped strings beginning with character 'A' together? You seem to expect that the sort algorithm takes characters from left to right, and when it compares 'A' and 'a', it will sort the

Re: Unicode Variation Selector and Combining character

2022-06-01 Thread Daniel Verite
Thomas Munro wrote: > Looking around a bit, it might be interesting to check if the > icu_character_boundaries() function in Daniel Vérité's icu_ext treats > IVSs as single grapheme clusters. It does. with strings(s) as ( values (U&'\+0066FE' || U&'\+0E0103'), (U&'\+00304B' || U

Re: Collation version tracking for macOS

2022-06-08 Thread Daniel Verite
Tom Lane wrote: > Yeah, and it's exactly at the level of quirks that things are likely > to change. Nobody's going to suddenly start sorting B before A. > They might, say, change their minds about where the digram "cz" > sorts relative to single letters, in languages where special rules >

Re: Add header support to text format and matching feature

2022-06-15 Thread Daniel Verite
Julien Rouhaud wrote: > Maybe that's just me but I understand "not supported" as "this makes > sense, but this is currently a limitation that might be lifted > later". Looking at ProcessCopyOptions(), there are quite a few invalid combinations of options that produce ERRCODE_FEATURE_NOT_

Re: Question about user/database-level parameters

2022-08-04 Thread Daniel Verite
Japin Li wrote: > However, if the database is in production, we cannot go into single-user > mode, should we provide an option to change this behavior on the fly? It already exists, through PGOPTIONS, which appears to work for local_preload_libraries, in a quick test. That is, you can lo

Re: [WIP] UNNEST(REFCURSOR): allowing SELECT to consume data from a REFCURSOR

2021-07-29 Thread Daniel Verite
Hi, Trying the v7a patch, here are a few comments: * SIGSEGV with ON HOLD cursors. Reproducer: declare c cursor with hold for select oid,relname from pg_class order by 1 limit 10; select * from rows_in('c') as x(f1 oid,f2 name); consumes a bit of time, then crashes and generates a 13 GB

Re: very long record lines in expanded psql output

2021-08-05 Thread Daniel Verite
Platon Pronko wrote: > Maybe we can avoid making the header line longer than terminal width > for \pset border 0 and \pset border 1? We already have terminal > width calculated. Please see attached a patch with the proposed > implementation. +1 for doing something against these long lines

Re: Built-in CTYPE provider

2024-07-05 Thread Daniel Verite
Noah Misch wrote: > > I don't think the builtin locale provider is any different in this respect > > from the other providers: The locale data might change and there is a > > version mechanism to track that. We don't prevent pg_upgrade in scenarios > > like that for other providers. > >

Re: Support LIKE with nondeterministic collations

2024-04-30 Thread Daniel Verite
Peter Eisentraut wrote: > This patch adds support for using LIKE with nondeterministic > collations. So you can do things such as > > col LIKE 'foo%' COLLATE case_insensitive Nice! > The pattern is partitioned into substrings at wildcard characters > (so 'foo%bar' is partitioned i

Re: Support LIKE with nondeterministic collations

2024-05-03 Thread Daniel Verite
Peter Eisentraut wrote: > Yes, certainly, and there is also no indexing support (other than for > exact matches). The ICU docs have this note about prefix matching: https://unicode-org.github.io/icu/userguide/collation/architecture.html#generating-bounds-for-a-sort-key-prefix-matching

<    1   2   3   4   >