Re: ICU for global collation

2022-03-15 Thread Daniel Verite
Julien Rouhaud wrote: > > > While on that topic, the doc should probably mention that default ICU > > > collations can only be deterministic. > > > > Well, there is no option to do otherwise, so I'm not sure where/how to > > mention that. We usually don't document options that don't exist

Re: ICU for global collation

2022-03-15 Thread Daniel Verite
Finnerty, Jim wrote: > In ICU, the "locale" is just the first part of what we can pass to the > "locale" parameter in CREATE COLLATION - the part before the optional '@' > delimiter. The ICU locale does not include the secondary or tertiary > properties, Why not? Please see https://unico

Re: popcount

2020-12-30 Thread Daniel Verite
David Fetter wrote: +Datum +byteapopcount(PG_FUNCTION_ARGS) +{ + bytea *t1 = PG_GETARG_BYTEA_PP(0); + int len, result; + + len = VARSIZE_ANY_EXHDR(t1); + result = pg_popcount(VARDATA_ANY(t1), len); + + PG_RETURN_INT32(result); +} The input may h

Re: PATCH: Batch/pipelining support for libpq

2020-11-14 Thread Daniel Verite
Alvaro Herrera wrote: > I adapted the test code to our code style. I also removed the "timings" > stuff; I think that's something better left to pgbench. > > (I haven't looked at Daniel's pgbench stuff yet, but I will do that > next.) The patch I posted in [1] was pretty simple, but at

Re: PATCH: Batch/pipelining support for libpq

2020-11-23 Thread Daniel Verite
Hi, Here's a new version with the pgbench support included. Best regards, -- Daniel Vérité PostgreSQL-powered mailer: https://www.manitou-mail.org Twitter: @DanielVerite diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 9ce32fb39b..2a94f8f6b9 100644 --- a/doc/src/sgml/libpq.s

Re: popcount

2021-01-18 Thread Daniel Verite
Peter Eisentraut wrote: > + /* > +* ceil(VARBITLEN(ARG1)/BITS_PER_BYTE) > +* done to minimize branches and instructions. > +*/ > > I don't know what that is supposed to mean or why that kind of tuning > would be necessary for a user-callable function. Also, the formula jus

Re: Are datcollate/datctype always libc even under --with-icu ?

2021-12-23 Thread Daniel Verite
Chapman Flack wrote: > Next question: the "currently" in that comment suggests that could change, > but is there any present intention to change it, or is this likely to just > be the way it is for the foreseeable future? Some related patches and discussions: * ICU as default collation p

Re: ICU for global collation

2022-01-07 Thread Daniel Verite
Julien Rouhaud wrote: > If you want a database with an ICU default collation the lc_collate > and lc_ctype should inherit what's in the template database and not > what was provided in the LOCALE I think. You could still probably > overload them in some scenario, but without a list of wha

Re: ICU for global collation

2022-01-10 Thread Daniel Verite
Peter Eisentraut wrote: > Unlike in the previous patch, where the ICU > collation name was written in datcollate, there is now a third column > (daticucoll), so we can store all three values. I think some users would want their db-wide ICU collation to be case/accent-insensitive. Postgr

Re: ICU for global collation

2022-01-10 Thread Daniel Verite
Julien Rouhaud wrote: > > The lack of these fields overall suggest the idea that when CREATE > > DATABASE is called with a global ICU collation, what if it somehow > > inserted the collation into pg_collation in the new database? > > Then pg_database would just store the collation oid and

Re: ICU for global collation

2022-01-11 Thread Daniel Verite
Julien Rouhaud wrote: > > I guess there's still the possibility of requiring that the ICU db-wide > > collation of the new database does exist in the template database, > > and then the CREATE DATABASE would refer to that collation instead of > > an independent locale string. > > That cou

Re: pgsql: Add libpq pipeline mode support to pgbench

2021-03-17 Thread Daniel Verite
Fabien COELHO wrote: > For consistency with the existing \if … \endif, ISTM that it could have > been named \batch … \endbatch or \pipeline … \endpipeline? "start" mirrors "end". To me, the analogy with \if-\endif is not obvious. Grammatically \if is meant to introduce the expression aft

Re: insensitive collations

2021-03-25 Thread Daniel Verite
Jim Finnerty wrote: > For a UTF8 encoded, case-insensitive (nondeterministic), accent-sensitive > ICU > collation, a LIKE predicate can be used with a small transformation of the > predicate, and the pattern can contain multi-byte characters: > > from: > > SELECT * FROM locations WHERE l

Re: insensitive collations

2021-03-25 Thread Daniel Verite
Jim Finnerty wrote: > Currently nondeterministic collations are disabled at the database level. Deterministic ICU collations are also disabled. > The cited reason was because of the lack of LIKE support and because certain > catalog views use LIKE. But the catalogs shouldn't use the d

Re: Calendar support in localization

2021-03-26 Thread Daniel Verite
Thomas Munro wrote: > Right, so if this is done by trying to extend Daniel Verite's icu_ext > extension (link given earlier) and Robert's idea of a fast-castable > type, I suppose you might want now()::icu_date + '1 month'::internal > to advance you by one Ethiopic month if you have done S

Re: Calendar support in localization

2021-03-29 Thread Daniel Verite
Matthias van de Meent wrote: > The results for the Japanese locale should be "0003 Reiwa" instead of > "0033 Heisei", as the era changed in 2019. ICU releases have since > implemented this and other corrections; this specific change was > implemented in the batched release of ICU versions

Re: Calendar support in localization

2021-03-30 Thread Daniel Verite
Surafel Temesgen wrote: > > About intervals, if there were locale-aware functions like > > add_interval(timestamptz, interval [, locale]) returns timestamptz > > or > > sub_timestamp(timestamptz, timestamptz [,locale]) returns interval > > that would use ICU to compute the results accord

Re: insensitive collations

2021-04-03 Thread Daniel Verite
Jim Finnerty wrote: > SET client_encoding = WIN1252; > [...] > postgres=# SELECT * FROM locations WHERE location LIKE 'Franche-Comt__'; -- > the wildcard is applied byte by byte instead of character by character, so > the 2-byte accented character is matched only by 2 '_'s >location

Re: Add header support to text format and matching feature

2020-07-25 Thread Daniel Verite
Rémi Lapeyre wrote: > Here’s a new version that fix all the issues. Here's a review of v4. The patch improves COPY in two ways: - COPY TO and COPY FROM now accept "HEADER ON" for the TEXT format (previously it was only for CSV) - COPY FROM also accepts "HEADER match" to tell that ther

EDB builds Postgres 13 with an obsolete ICU version

2020-08-03 Thread Daniel Verite
Hi, As a follow-up to bug #16570 [1] and other previous discussions on the mailing-lists, I'm checking out PG13 beta for Windows from: https://www.enterprisedb.com/postgresql-early-experience and it ships with the same obsolete ICU 53 that was used for PG 10,11,12. Besides not having the latest

Re: Re: csv format for psql

2018-03-22 Thread Daniel Verite
David Steele wrote: > Do you know when you'll have an updated patch that addresses the minor > issues brought up in review and the concern above? Here's an update addressing the issues discussed: - fieldsep and recordsep are used, no more fieldsep_csv - the command line option is --csv w

Re: Re: csv format for psql

2018-03-23 Thread Daniel Verite
Pavel Stehule wrote: > It should not be hard. All formats can has '|' like now, and csv can have a > ',' - then if field separator is not explicit, then default field separator > is used, else specified field separator is used. > > You can see my idea in attached patch With that patch, c

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2018-03-23 Thread Daniel Verite
Kyotaro HORIGUCHI wrote: > A disucssion on psql batch mode was held in another branch of > this thread. How do we treat that? There's a batch mode for pgbench in a patch posted in [1], with \beginbatch and \endbatch commands, but nothing for psql AFAICS. psql is more complicated because c

Re: Re: csv format for psql

2018-03-26 Thread Daniel Verite
Pavel Stehule wrote: > implemented in attached patch Consider your initial output of \pset, when no option is specified = $ ./psql psql (11devel) Type "help" for help. postgres=# \pset border 1 columns 0 expanded off fie

Re: Re: csv format for psql

2018-03-26 Thread Daniel Verite
Pavel Stehule wrote: > b) the list of pset options is bloating - every possible new format can > introduce fieldsep_X option What new format? The usefulness of fieldsep does not extend outside of xSV, and it's no suprise that there have been no other use for a fieldsep-like variable until

Re: csv format for psql

2018-03-29 Thread Daniel Verite
David G. Johnston wrote: > Could someone post how captions, rows-only, and footer pset settings factor > into this? Specifically are they fixed to on/off or will they hide/show if > users request them explicitly? This is described in the doc with: + csv format writes columns se

Re: csv format for psql

2018-03-29 Thread Daniel Verite
David G. Johnston wrote: > Or, really, just make --csv take an optional argument which, if present, is > the delimiter. I don't think psql can support optional arguments because psql --option foo would be ambiguous, since foo could be the option's value or the name of a database to connec

Re: csv format for psql

2018-03-29 Thread Daniel Verite
Isaac Morland wrote: > The actual reason I'm posting this is because some of the discussion has > made me a bit confused: there is already a CSV format defined for the COPY > command and used by the psql \copy. I just want to check that what is being > discussed here would use the exact sa

Re: csv format for psql

2018-03-29 Thread Daniel Verite
David G. Johnston wrote: > Unaligned format could grow its own fieldsep if it wanted to but it can > also just use the default provided fieldsep var whose default value is > pipe. If it did grow one it would need to understand "not set" in order to > preserve existing behavior. We don't

Re: csv format for psql

2018-04-07 Thread Daniel Verite
Pavel Stehule wrote: > so we can to have > > \pset format xxx > > and set of local possibly changed defaults > > \pset csv_fieldsep , > \pset csv_tuplesonly on > \pset unaligned_fieldsep | > \pset unaligned_tuplesonly off tuples_only (\t) is a single setting that is currently used by a

Re: csv format for psql

2018-04-07 Thread Daniel Verite
Peter Eisentraut wrote: > Another thought: Isn't CSV just the same as unaligned output plus some > quoting? Could we add a quote character setting and then define --csv > to be quote-character = " and fieldsep = , ? Plus footer set to off. So --csv would be like \pset format unaligned \p

Re: csv format for psql

2018-04-07 Thread Daniel Verite
Isaac Morland wrote: > OK, mostly trying to avoid commenting because I doubt I have much to add. > But. If I ask for CSV and don't specify any overrides, I expect to get > "C"omma separated values, not some other character. More specifically, if I > say --csv I expect to get files that are

Re: csv format for psql

2018-04-07 Thread Daniel Verite
Daniel Verite wrote: > The output might still differ compared to COPY in that line endings > depend on the client-side OS. There's also the minor issue > of a single \ by itself on a line, which gets quoted by COPY > and not by psql. I meant \. or backslash followe

Re: Proposal: Adding json logging

2018-04-16 Thread Daniel Verite
David Arnold wrote: > Not claiming this assumption does imply parsing of a *rolling* set > of log lines with *previously unkown cardinality*. That's expensive > on computing resources. I don't have actual numbers, but it doesn't > seem too far fetched, neither. > I filed a question to the

Re: Proposal: Adding json logging

2018-04-17 Thread Daniel Verite
David Arnold wrote: > Interesting, does that implicitly mean the whole log event would get > transmitted as a "line" (with CRLF) in CSV. To me it's implied by the doc at: https://www.postgresql.org/docs/current/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG > In the aff

Re: [PATCH] vacuumlo: print the number of large objects going to be removed

2019-09-06 Thread Daniel Verite
Michael Paquier wrote: > Sure. However do we need to introduce this much complication as a > goal for this patch though whose goal is just to provide hints about > the progress of the work done by vacuumlo? Yeah, I went off on a tangent when realizing that ~500 lines of C client-side co

Create collation reporting the ICU locale display name

2019-09-11 Thread Daniel Verite
Hi, The 'locale' or 'lc_collate/lc_ctype' argument of an ICU collation may have a complicated syntax, especially with non-deterministic collations, and input mistakes in these names will not necessarily be detected as such by ICU. The "display name" of a locale is a simple way to get human-reada

Re: Create collation reporting the ICU locale display name

2019-09-12 Thread Daniel Verite
Michael Paquier wrote: > On Wed, Sep 11, 2019 at 04:53:16PM +0200, Daniel Verite wrote: > > I think it would be nice to have CREATE COLLATION report this > > information as feedback in the form of a NOTICE message. > > PFA a simple patch implementing that. > >

Re: Create collation reporting the ICU locale display name

2019-09-13 Thread Daniel Verite
Michael Paquier wrote: > Or could it make sense to provide a system function which returns a > collation description for at least an ICU-provided one? We could make > use of that in psql for example. If we prefer having a function over the instant feedback effect of the NOTICE, the funct

Re: Create collation reporting the ICU locale display name

2019-09-14 Thread Daniel Verite
Tom Lane wrote: > I think that's a useful function, but it's a different function from > the one first proposed, which was to tell you the properties of a > collation you already installed (which might not be ICU, even). > Perhaps we should have both. The pre-create use case would look l

Re: Create collation reporting the ICU locale display name

2019-09-14 Thread Daniel Verite
Tom Lane wrote: > > This output tend to reveal mistakes with tags, which is why I thought > > to expose it as a NOTICE. It addresses the case of a user > > who wouldn't suspect an error, so the "in-your-face" effect is > > intentional. With the function approach, the user must be > > proac

Re: ICU for global collation

2019-09-17 Thread Daniel Verite
Hi, When trying databases defined with ICU locales, I see that backends that serve such databases seem to have their LC_CTYPE inherited from the environment (as opposed to a per-database fixed value). That's a problem for the backend code that depends on libc functions that themselves depend on

Re: insensitive collations

2019-03-04 Thread Daniel Verite
Peter Eisentraut wrote: [v7-0001-Collations-with-nondeterministic-comparison.patch] +GenericMatchText(const char *s, int slen, const char *p, int plen, Oid collation) { + if (collation && !lc_ctype_is_c(collation) && collation != DEFAULT_COLLATION_OID) + { +pg_locale_tlocale =

Re: insensitive collations

2019-03-05 Thread Daniel Verite
Peter Eisentraut wrote: > Older ICU versions (<54) don't support all the locale customization > options, so many of my new tests in collate.icu.utf8.sql will fail on > older systems. What should we do about that? Have another extra test file? Maybe stick to the old-style syntax for the

Re: insensitive collations

2019-03-07 Thread Daniel Verite
Peter Eisentraut wrote: > The problem is not the syntax but that the older ICU versions don't > support the *functionality* of ks-level2 or colStrength=secondary. If > you try it, you will simply get a normal case-sensitive behavior. My bad, I see now that the "old locale extension synta

Re: Willing to fix a PQexec() in libpq module

2019-03-19 Thread Daniel Verite
Tom Lane wrote: > Unfortunately, if the default behavior doesn't change, then there's little > argument for doing this at all. The security reasoning behind doing > anything in this area would be to provide an extra measure of protection > against SQL-injection attacks on carelessly-writt

Re: psql - add SHOW_ALL_RESULTS option

2019-07-24 Thread Daniel Verite
Fabien COELHO wrote: > >> I'd go further and suggest that there shouldn't be a variable > >> controlling this. All results that come in should be processed, period. > > > > I agree with that. > > I kind of agree as well, but I was pretty sure that someone would complain > if the current

Re: psql - add SHOW_ALL_RESULTS option

2019-07-25 Thread Daniel Verite
Fabien COELHO wrote: > Attached a "do it always version", which does the necessary refactoring. > There is seldom new code, it is rather moved around, some functions are > created for clarity. Thanks for the update! FYI you forgot to remove that bit: --- a/src/bin/psql/tab-complete.c +

Re: psql - add SHOW_ALL_RESULTS option

2019-07-26 Thread Daniel Verite
Fabien COELHO wrote: > sh> /usr/bin/psql > psql (12beta2 ...) > fabien=# \set FETCH_COUNT 2 > fabien=# SELECT 1234 \; SELECT 5432 ; > fabien=# > > same thing with pg 11.4, and probably down to every version of postgres > since the feature was implemented... > > I think that

Re: [HACKERS] proposal: psql command \graw

2018-01-27 Thread Daniel Verite
Pavel Stehule wrote: > We are able to generate html/tex/markdown formats on client side. CSV is > more primitive, but much more important data format, so it should not be a > problem. But I remember some objections related to code duplication. While experimenting with adding csv as an out

csv format for psql

2018-01-30 Thread Daniel Verite
Hi, This patch implements csv as an output format in psql (\pset format csv). It's quite similar to the unaligned format, except that it applies CSV quoting rules (obviously!) and that it prints no footer and no title. As with unaligned, a header with column names is output unless tuples_only is

Re: csv format for psql

2018-01-31 Thread Daniel Verite
Fabien COELHO wrote: > > This patch implements csv as an output format in psql > > (\pset format csv). > > Would you consider registering it in the next CF? Done here: https://commitfest.postgresql.org/17/1500/ Best regards, -- Daniel Vérité PostgreSQL-powered mailer: http://www.manit

Re: csv format for psql

2018-01-31 Thread Daniel Verite
Pavel Stehule wrote: > This format is too important, so some special short or long option can be > practical (it will be printed in help) > > some like --csv I guess -C/--csv could be used, like there already is -H/--html. > I found one issue - PostgreSQL default field separator is "|".

Re: proposal: alternative psql commands quit and exit

2018-02-01 Thread Daniel Verite
Bruce Momjian wrote: > One open issue is the existing help display is inaccurate on Windows: > > Use \\? for help or press control-C to clear the input buffer. ! #ifndef WIN32 ! puts(_("Use control-D to quit.")); ! #else ! puts(_("Use control-C to quit.")); ! #endif But Cont

Re: proposal: alternative psql commands quit and exit

2018-02-01 Thread Daniel Verite
Bruce Momjian wrote: > > Also, the fact that Control-D can quit in the middle of a > > multiline query without any confirmation is a usability problem, because > > you can always fat-finger a Ctrl+key. By comparison, bash doesn't > > accept it and emits the same error as if a script was i

Re: csv format for psql

2018-02-12 Thread Daniel Verite
Pavel Stehule wrote: > > I guess -C/--csv could be used, like there already is -H/--html. > > > > I prefer just long option only. Maybe in future we can use short "C" for > something else better. There is only few free short commands. Looking at parse_psql_options(), currently psql has 3

Re: csv format for psql

2018-02-12 Thread Daniel Verite
Pavel Stehule wrote: > some like "\pset format csv , header_off If we did that, we'd need to reconsider the interactions of this with \t on|off and \pset footer on|off and how to keep things consistent with the unaligned format. I feel like it's not worth the trouble. We can still add thi

Re: ICU for global collation

2019-11-01 Thread Daniel Verite
Peter Eisentraut wrote: > I looked into this problem. The way to address this would be adding > proper collation support to the text search subsystem. See the TODO > markers in src/backend/tsearch/ts_locale.c for starting points. These > APIs spread out to a lot of places, so it will

Re: updating unaccent.rules for Arabic letters

2019-11-04 Thread Daniel Verite
kerbrose khaled wrote: > I would like to update unaccent.rules file to support Arabic letters. so > could someone help me or tell me how could I add such contribution. I > attached the file including the modifications, only the last 4 lines. The Arabic letters are found in the Unicode blo

Making psql error out on output failures

2019-12-16 Thread Daniel Verite
Hi, When exporting data with psql -c "..." >file or select ... \g file inside psql, post-creation output errors are silently ignored. The problem can be seen easily by creating a small ramdisk and filling it over capacity: $ sudo mount -t tmpfs -o rw,size =1M tmpfs /mnt/ramdisk $ psql -d postg

Re: Unicode normalization SQL functions

2020-01-06 Thread Daniel Verite
Peter Eisentraut wrote: > Also, there is a way to optimize the "is normalized" test for common > cases, described in UTR #15. For that we'll need an additional data > file from Unicode. In order to simplify that, I would like my patch > "Add support for automatically updating Unicode

Re: [WIP] UNNEST(REFCURSOR): allowing SELECT to consume data from a REFCURSOR

2020-01-09 Thread Daniel Verite
Dent John wrote: > I’ve made a revision of this patch. Some comments: * the commitfest app did not extract up the patch from the mail, possibly because it's buried in the MIME structure of the mail (using plain text instead of HTML messages might help with that). The patch has no status

Re: [WIP] UNNEST(REFCURSOR): allowing SELECT to consume data from a REFCURSOR

2020-01-10 Thread Daniel Verite
Dent John wrote: > Yes. That’s at least true if unnest(x) is used in the FROM. If it’s used in > the SELECT, actually it can get the performance benefit right away At a quick glance, I don't see it called in the select-list in any of the regression tests. When trying it, it appears to cr

Re: Making psql error out on output failures

2020-01-14 Thread Daniel Verite
David Z wrote: > $ psql -d postgres -At -c "select repeat('111', 100)" > > /mnt/ramdisk/file The -A option selects the unaligned output format and -t switches to the "tuples only" mode (no header, no footer). > Test-2: delete the "file", run the command within psql console, > $ rm /

Re: [WIP] UNNEST(REFCURSOR): allowing SELECT to consume data from a REFCURSOR

2020-01-14 Thread Daniel Verite
Dent John wrote: > It’s crashing when it’s checking that the returned tuple matches the > declared return type in rsinfo->setDesc. Seems rsinfo->setDesc gets > overwritten. So I think I have a memory management problem. What is the expected result anyway? A single column with a "record" t

Re: Making psql error out on output failures

2020-01-16 Thread Daniel Verite
David Zhang wrote: > If I change the error log message like below, where "%m" is used to pass the > value of strerror(errno), "could not write to output file:" is copied from > function "WRITE_ERROR_EXIT". > - pg_log_error("Error printing tuples"); > +

Re: psql - add SHOW_ALL_RESULTS option

2020-01-17 Thread Daniel Verite
Alvaro Herrera wrote: > if this patch enables other psql features, it might be a good step > forward. Yes. For instance if the stored procedures support gets improved to produce several result sets, how is psql going to benefit from it while sticking to the old way (PGresult *r = PQexec(q

Re: psql - add SHOW_ALL_RESULTS option

2020-01-17 Thread Daniel Verite
Tom Lane wrote: > I'm not really holding my breath for that to happen, considering > it would involve fundamental breakage of the wire protocol. > (For example, extended query protocol assumes that Describe > Portal only needs to describe one result set. There might be > more issues, but

Re: Making psql error out on output failures

2020-01-20 Thread Daniel Verite
David Zhang wrote: > Yes, I agree with you. For case 2 "select repeat('111', 100) \g > /mnt/ramdisk/file", it can be easily fixed with more accurate error > message similar to pg_dump, one example could be something like below. > But for case 1 "psql -d postgres -At -c "select repe

Re: making the backend's json parser work in frontend code

2020-01-23 Thread Daniel Verite
Robert Haas wrote: > With the format I proposed, you only have to worry that the > file name might contain a tab character, because in that format, tab > is the delimiter It could be CSV, which has this problem already solved, is easier to parse than JSON, certainly no less popular, and i

Re: Unicode normalization SQL functions

2020-01-28 Thread Daniel Verite
Peter Eisentraut wrote: > Here is an updated patch set that now also implements the "quick check" > algorithm from UTR #15 for making IS NORMALIZED very fast in many cases, > which I had mentioned earlier in the thread. I found a bug in unicode_is_normalized_quickcheck() which is trigge

Re: Making psql error out on output failures

2020-01-28 Thread Daniel Verite
David Zhang wrote: > The error "No space left on device" can be logged by fprintf() which is > redefined as pg_fprintf() when print_aligned_text() is called Are you sure? I don't find that redefinition. Besides print_aligned_text() also calls putc and puts. > Will that be a better solut

Re: Changes to pg_dump/psql following collation "C" in the catalog

2019-04-04 Thread Daniel Verite
Tom Lane wrote: > "Daniel Verite" writes: > > One consequence of using the "C" collation in the catalog versus > > the db collation is that pg_dump -t with a regexp may not find > > the same tables as before. It happens when these conditions ar

Re: Changes to pg_dump/psql following collation "C" in the catalog

2019-04-05 Thread Daniel Verite
Chapman Flack wrote: > >> "Daniel Verite" writes: > >>> One consequence of using the "C" collation in the catalog versus > >>> the db collation > > As an intrigued Person Following At Home, I was happy when I found > t

Re: Changes to pg_dump/psql following collation "C" in the catalog

2019-04-05 Thread Daniel Verite
Tom Lane wrote: > Hm, if that's as much as we have to touch, I think there's a good > argument for squeezing it into v12 rather than waiting. The point > here is mostly to avoid a behavior change from pre-v12 Yes. I was mentioning the next CF because ISTM that nowadays non-committers are

Re: Changes to pg_dump/psql following collation "C" in the catalog

2019-04-06 Thread Daniel Verite
Tom Lane wrote: > > PFA a new version adding the clause for only 12 and up, since the > > previous versions are not concerned, and as you mention, really old > > versions would fail otherwise. > > Pushed with some fiddling with the comments, and changing the collation > names to be schema

Re: Cleanup/remove/update references to OID column

2019-04-10 Thread Daniel Verite
Justin Pryzby wrote: > Cleanup/remove/update references to OID column... > > ..in wake of 578b229718e8f. Just spotted a couple of other references that need updates: #1. In catalogs.sgml: attnum int2 The number of the column. Ordinary columns are

Re: PostgreSQL pollutes the file system

2019-04-13 Thread Daniel Verite
Andreas Karlsson wrote: > The Debian packagers already use pg_createcluster for their script which > wraps initdb, and while pg_initdb is a bit misleading (it creates a > cluster rather than a database) it is not that bad. But that renaming wouldn't achieve anything in relation to the s

Re: Cleanup/remove/update references to OID column

2019-04-15 Thread Daniel Verite
Andres Freund wrote: > Yes, I was planning to commit that soon-ish. There still seemed > review / newer versions happening, though, so I was thinking of waiting > for a bit longer. You might want to apply this trivial one in the same batch: index 452f307..7cfb67f 100644 --- a/src/bin/pg_

Trouble with FETCH_COUNT and combined queries in psql

2019-04-22 Thread Daniel Verite
Hi, When FETCH_COUNT is set, queries combined in a single request don't work as expected: \set FETCH_COUNT 10 select pg_sleep(2) \; select 1; No result is displayed, the pg_sleep(2) is not run, and no error is shown. That's disconcerting. The sequence that is sent under the hood is: #1 BEG

Re: Trouble with FETCH_COUNT and combined queries in psql

2019-04-23 Thread Daniel Verite
Fabien COELHO wrote: > I added some stuff to extract embedded "\;" for pgbench "\cset", which has > been removed though, but it is easy to add back a detection of "\;", and > also to detect select. If the position of the last select is known, the > cursor can be declared in the right pl

Re: Trouble with FETCH_COUNT and combined queries in psql

2019-04-23 Thread Daniel Verite
Tom Lane wrote: > Keep in mind that a large part of the reason why the \cset patch got > bounced was exactly that its detection of \; was impossibly ugly > and broken. Don't expect another patch using the same logic to > get looked on more favorably. Looking at the end of the discussion

RE: psql - add SHOW_ALL_RESULTS option

2019-05-15 Thread Daniel Verite
Fabien COELHO wrote: > >> IMHO this new setting should be on by default: few people know about \; so > >> it would not change anything for most, and I do not see why those who use > >> it would not be interested by the results of all the queries they asked > >> for. > > I agree with your

Re: COPY FROM WHEN condition

2018-11-02 Thread Daniel Verite
Pavel Stehule wrote: > > SELECT x.a, sum(x.b) > > FROM ( COPY INLINE '/path/to/foo.txt' FORMAT CSV ) as x( a integer, b > > numeric, c text, d date, e json) ) > > WHERE x.d >= '2018-11-01' > > > > > Without some special feature this example is not extra useful. It is based > on copy on ser

Re: csv format for psql

2018-11-02 Thread Daniel Verite
Michael Paquier wrote: > If you can produce a new version, please feel free to post it. Here's a rebased version with a couple regression tests added per the discussions during the previous CF. Now at https://commitfest.postgresql.org/20/1861/ Best regards, -- Daniel Vérité PostgreSQL

Re: COPY FROM WHEN condition

2018-11-03 Thread Daniel Verite
David Fetter wrote: > It also seems like a violation of separation of concerns to couple > FEBE to grammar, so there'd need to be some way to do those things > separately, too. After re-reading psql/copy.c, I withdraw what I said upthread: it doesn't appear necessary to add anything to su

Re: csv format for psql

2018-11-06 Thread Daniel Verite
Michael Paquier wrote: > Ordering them in alphabetical order is a good idea due to the high > number of options available, and more would pile up even if this > separates a bit "aligned" and "unaligned", so I have have separated > those diffs from the core patch and committed it, leaving t

Doc patch on psql output formats

2018-11-06 Thread Daniel Verite
Hi, psql's documentation has this mention about output formats: "Unique abbreviations are allowed. (That would mean one letter is enough.)" but "one letter is enough" is not true since 9.3 that added "latex-longtable" sharing the same start as "latex", and then 9.5 added "asciidoc" with the s

Re: Doc patch on psql output formats

2018-11-06 Thread Daniel Verite
Tom Lane wrote: > > When a non-unique abbreviation is used, psql uses the first > > match in an arbitrary order defined in do_pset() by > > a cascade of pg_strncasecmp(). > > Ugh. Should we not fix the code so that it complains if there's > not a unique match? I would bet that the code

Re: csv format for psql

2018-11-07 Thread Daniel Verite
Michael Paquier wrote: > -C could also be useful for other things, say compression. The patch does not grab any short letter, as the consensus went that way. > + pset.popt.topt.fieldSepCsv = pg_strdup(","); > Let's store that in a variable instead of hardcoding it. I guess it could go

Re: csv format for psql

2018-11-09 Thread Daniel Verite
Michael Paquier wrote: > +/* check for value being non-empty and with an MB length of 1 */ > +if (*value == '\0' || value[PQmblen(value, pset.encoding)] != '\0') > > It seems to me that this can just be replaced with that: > if (strlen(value) != 1) The above is meant to accept a

Alternative to \copy in psql modelled after \g

2018-11-09 Thread Daniel Verite
Hi, Currently \copy cannot span multiple lines (like any meta-command) and cannot use psql variables whereas \g can do both. The POC patch attached implements two meta-commands \copyfrom and \copyto that are to COPY what \g is to any other query: - they take the COPY query already var-expanded

Re: csv format for psql

2018-11-09 Thread Daniel Verite
Michael Paquier wrote: > Still what's the point except complicating the code? We don't care > about anything fancy in the backend-side ProcessCopyOptions() when > checking cstate->delim, and having some consistency looks like a good > thing to me. The backend has its reasons that don't a

Re: Alternative to \copy in psql modelled after \g

2018-11-09 Thread Daniel Verite
David G. Johnston wrote: > Do I understand correctly that you are proposing a slightly less > verbose alternative of: > > \o :filename > COPY TO STDOUT > \o Not strictly the same because of this: \o or \out [ filename ] “Query results” includes all tables, command responses, and

Re: Alternative to \copy in psql modelled after \g

2018-11-09 Thread Daniel Verite
David G. Johnston wrote: > POLA, for me at least, would be for \g [filename] to do exactly what > you are describing with the \copyto feature. I admit that if we could improve \g to handle COPY, it would be more elegant than the current proposal adding two meta-commands. But the copy-wor

Re: csv format for psql

2018-11-11 Thread Daniel Verite
Michael Paquier wrote: > - The experience is confusing, as the psql format uses different options > than the backend to do the same things: > -- tuples_only instead of HEADER. > -- fieldsep_csv instead of DELIMITER > -- null is an equivalent of the one with the same name, which is > actual

Re: Doc patch on psql output formats

2018-11-14 Thread Daniel Verite
Tom Lane wrote: > > but "one letter is enough" is not true since 9.3 that added > > "latex-longtable" sharing the same start as "latex", and then > > 9.5 added "asciidoc" with the same first letter as "aligned". > > Yeah, that text has clearly outstayed its welcome. > > > When a non-uni

Re: Doc patch on psql output formats

2018-11-15 Thread Daniel Verite
Tom Lane wrote: > Pushed. (I simplified the code a bit by using just one state variable, > and also made the error message more verbose.) Thanks! Best regards, -- Daniel Vérité PostgreSQL-powered mailer: http://www.manitou-mail.org Twitter: @DanielVerite

Re: csv format for psql

2018-11-26 Thread Daniel Verite
Tom Lane wrote: > And, in fact, right now *none* of psql's table output formats is both > unambiguous and reasonably simple/popular to use. So the astonishing > thing about this patch, IMO, is that we didn't do it a decade ago. Yeah, that's what motivated this submission in the first pla

Re: Doc patch on psql output formats

2018-11-26 Thread Daniel Verite
Tom Lane wrote: > As of HEAD, it's impossible to select latex output format > at all: > > regression=# \pset format latex > \pset: ambiguous abbreviation "latex" matches both "latex" and > "latex-longtable" Oops! > We could fix that by adding a special case to accept an exact match > im

  1   2   3   4   >