Re: [HACKERS] Dynamic result sets from procedures

2017-11-04 Thread Daniel Verite
Peter Eisentraut wrote: > > CREATE PROCEDURE test() > > LANGUAGE plpgsql > > AS $$ > > RETURN QUERY EXECUTE 'SELECT 1 AS col1, 2 AS col2'; > > END; > > $$; > > > > Or is that not possible or not desirable? > > RETURN means the execution ends there, so how would you return multiple

Re: [HACKERS] Dynamic result sets from procedures

2017-11-02 Thread Daniel Verite
Peter Eisentraut wrote: > CREATE PROCEDURE pdrstest1() > LANGUAGE SQL > AS $$ > DECLARE c1 CURSOR WITH RETURN FOR SELECT * FROM cp_test2; > DECLARE c2 CURSOR WITH RETURN FOR SELECT * FROM cp_test3; > $$; > > CALL pdrstest1(); > > and that returns those two result sets to the client. If

Re: [HACKERS] psql: new help related to variables are not too readable

2017-09-09 Thread Daniel Verite
Tomas Vondra wrote: > That's not what I meant. I've never felt a strong urge to avoid wrapping > at 80 chars when translating strings - simply translate in the most > meaningful way, if it gets longer than 80 chars and can't be easily > shortened, just wrap. If there are 60 or 80 character

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-09-04 Thread Daniel Verite
Tom Lane wrote: > Since we've blown past 80 columns on some of the other > output, maybe that doesn't matter. Or maybe we should shorten this > variable name so it doesn't force reformatting of all this text. The two-space left margin on the entire block does not add that much to readabi

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-22 Thread Daniel Verite
Andres Freund wrote: > > One option may be to leave that decision to the user by providing a > > PQBatchAutoFlush(true|false) property, along with a PQBatchFlush() > > function. > > What'd be the difference between PQflush() and PQbatchFlush()? I guess no difference, I was just not seein

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-22 Thread Daniel Verite
Andres Freund wrote: - if (pqFlush(conn) < 0) - goto sendFailed; + if (conn->batch_status == PQBATCH_MODE_OFF) + { + /* +* Give the data a push. In nonblock mode, don't complain if we're unable +* to send it all

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-22 Thread Daniel Verite
Craig Ringer wrote: > The kernel will usually do some packet aggregation unless we use > TCP_NODELAY (which we don't and shouldn't) Not sure. As a point of comparison, Oracle has it as a tunable parameter (TCP.NODELAY), and they changed its default from No to Yes starting from their 10g R

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-06-20 Thread Daniel Verite
Andres Freund wrote: > FWIW, I still think this needs a pgbench or similar example integration, > so we can actually properly measure the benefits. Here's an updated version of the patch I made during review, adding \beginbatch and \endbatch to pgbench. The performance improvement appears

Re: [HACKERS] Disallowing multiple queries per PQexec()

2017-06-15 Thread Daniel Verite
Andres Freund wrote: > Since it's an application writer's choice whether to use it, > it seems to make not that much sense to have a > serverside guc - it can't really be sensible set. The application writers who are concerned by this wouldn't know that they have a choice. If there were i

Re: [HACKERS] Disallowing multiple queries per PQexec()

2017-06-15 Thread Daniel Verite
Fabien COELHO wrote: > I'm not fully convinced by this feature: using multiple queries is a > useful trick to reduce network-related latency by combining several > queries in one packet. Devs and even ORMs could use this trick. It's proposed as an option. For apps that intentionally put

Re: [HACKERS] Disallowing multiple queries per PQexec()

2017-06-12 Thread Daniel Verite
Tom Lane wrote: > Bearing in mind that I'm not really for this at all... It's a band-aid, but certainly there are cases where a DBA confronted to a badly written website would just want to be able to: ALTER USER webuser SET allow_multiple_queries TO off; > But if an attacker is able to

Re: [HACKERS] Disallowing multiple queries per PQexec()

2017-06-12 Thread Daniel Verite
Surafel Temesgen wrote: > I modified the patch as such and added to commitfest 2017-07. A couple comments: + {"disallow_multiple_queries", PGC_POSTMASTER, CLIENT_CONN_OTHER, + gettext_noop("Disallow multiple queries per query string."), +

Re: [HACKERS] export import bytea from psql

2017-05-11 Thread Daniel Verite
Pavel Stehule wrote: > It is similar to my first or second proposal - rejected by Tom :( Doesn't it differ? ISTM that in the patches/discussion related to: https://commitfest.postgresql.org/11/787/ it was proposed to change \set in one way or another, and also in the point #1 of this pres

Re: [HACKERS] export import bytea from psql

2017-05-09 Thread Daniel Verite
Pavel Stehule wrote: > 1. using psql variables - we just need to write commands \setfrom > \setfrombf - this should be very simple implementation. The value can be > used more times. On second hand - the loaded variable can hold lot of > memory (and it can be invisible for user). This cou

Re: [HACKERS] PG 10 release notes

2017-04-27 Thread Daniel Verite
Fabien COELHO wrote: >Fix psql \p to always print what would be executed by \g or \w (Daniel >Vérité) > >Previously \p didn't properly print the reverted-to command after a >buffer contents reset. CLARIFY? > > The fix is linked to the change introduced by Tom when > refa

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-04-14 Thread Daniel Verite
Fabien COELHO wrote: > Pavel also suggested some support for TEXT, although I would like to see a > use case. That could be another extension to the engine. SQLSTATE is text. Also when issuing "psql -v someoption=value -f script", it's reasonable to want to compare :someoptionvar to 'so

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-04-03 Thread Daniel Verite
Fabien COELHO wrote: > Now it could be decided that \set in psql stays simplistic because it is > not needed as much as it is with pgbench. Fine with me. It's not just that. It's that currently, if we do in psql: \set d sqrt(1 + random(1000) * 17) then we get that: \echo :d sqrt(1+ran

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-04-03 Thread Daniel Verite
Hi, In interactive mode, the warning in untaken branches is misleading when \endif is on the same line as the commands that are skipped. For instance: postgres=# \if false \echo NOK \endif \echo command ignored; use \endif or Ctrl-C to exit current \if block postgres=# From the point of

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-04-03 Thread Daniel Verite
Tom Lane wrote: > I really dislike this syntax proposal. It would get us into having > to count nested brackets, and distinguish quoted from unquoted brackets, > and so on ... and for what? It's not really better than > > \if sql select 1 from pg_catalog.pg_extension where extname='pg

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-04-03 Thread Daniel Verite
Fabien COELHO wrote: > My 0.02 € about server-side expressions: ISTM that there is nothing > obvious/easy to do to include these: > > - how would it work, both with \set ... and \if ...? The idea is that the need to have two command (a SELECT .. \gset followed by an \if) and a tempora

Re: [HACKERS] Suggested fix for \p and \r in psql

2017-04-02 Thread Daniel Verite
Tom Lane wrote: > If we do phrase it like that, I think that the most natural behavior > for \r is the way I have it in HEAD --- you'd expect it to clear > the query buffer, but you would not expect it to forget the most > recent command. Okay, leaving out \r as it is and changing only \p

Re: [HACKERS] Suggested fix for \p and \r in psql

2017-04-02 Thread Daniel Verite
Tom Lane wrote: > > 1. \p ignores the "previous buffer". Example: > > Yeah, I did that intentionally, thinking that the old behavior was > confusing. We can certainly discuss it though. I'd tend to agree > with your point that \p and \w should print the same thing, but > maybe neither o

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-04-02 Thread Daniel Verite
Fabien COELHO wrote: > Note that this is already available indirectly, as show in the > documentation. > > SELECT some-boolean-expression AS okay \gset > \if :okay > \echo boolean expression was true > \else > \echo boolean expression was false > \endif Yes, the question

[HACKERS] Suggested fix for \p and \r in psql

2017-04-02 Thread Daniel Verite
Hi, I've noticed two issues with the query buffer post-commit e984ef5 (Support \if ... \elif ... \else ... \endif in psql scripting): 1. \p ignores the "previous buffer". Example: postgres=# select 1; ?column? -- 1 (1 row) postgres=# \p Query buffer is empty. That doesn't m

Re: [HACKERS] Variable substitution in psql backtick expansion

2017-03-31 Thread Daniel Verite
Tom Lane wrote: > Thoughts? ISTM that expr is too painful to use to be seen as the idiomatic way of achieving comparison in psql. Among its disadvantages, it won't work on windows, and its interface is hard to work with due to the necessary quoting of half its operators, and the mandator

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-30 Thread Daniel Verite
Vaishnavi Prabakaran wrote: > Hmm, With batch mode, after sending COPY command to server(and server > started processing the query and goes into COPY state) , client does not > immediately read the result , but it keeps sending other queries to the > server. By this time, server already en

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-28 Thread Daniel Verite
Michael Paquier wrote: > # Running: testlibpqbatch dbname=postgres 1 copyfailure > dispatching SELECT query failed: cannot queue commands during COPY > > COPYBUF: 5 > > Error status and message got from server due to COPY command failure > are : PGRES_FATAL_ERROR ERROR: unexpected m

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-27 Thread Daniel Verite
Vaishnavi Prabakaran wrote: > Am going to include the test which you shared in the test patch. Please let > me know if you want to cover anymore specific cases to gain confidence. One bit of functionality that does not work in batch mode and is left as is by the patch is the PQfn() fast p

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-27 Thread Daniel Verite
Vaishnavi Prabakaran wrote: > Please let me know if you think this is not enough but wanted to update > section 33.6 also? Yes, if the right place to call PQsetSingleRowMode() is immediately after PQbatchQueueProcess(), I think we need to update "33.6. Retrieving Query Results Row-By-Row"

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-17 Thread Daniel Verite
Tom Lane wrote: > OT_WHOLE_LINE is not what you want because that results in verbatim > copying, without variable expansion or anything But if we want to implement "\if defined :foo" in the future isn't it just what we need? Also we could leave open the option to accept an SQL expression

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-16 Thread Daniel Verite
Vaishnavi Prabakaran wrote: > So, attached the alternative fix for this issue. > Please share me your thoughts. I assume you prefer the alternative fix because it's simpler. > I would also like to hear Craig's opinion on it before applying this fix > to the original patch, just to make s

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-03-13 Thread Daniel Verite
Tom Lane wrote: > when we see \if is that we do nothing but absorb text > until we see the matching \endif. At that point we could bitch and throw > everything away if, say, there's \elif after \else, or anything else you > want to regard as a "compile time error". Otherwise we start exe

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-13 Thread Daniel Verite
Vaishnavi Prabakaran wrote: > > while (QbatchQueueProcess(conn)) { > >r = PQsetSingleRowMode(conn); > >if (r!=1) { > > fprintf(stderr, "PQsetSingleRowMode() failed"); > >} > >.. > Thanks for investigating the problem, and could you kindly exp

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-10 Thread Daniel Verite
Hi, I notice that PQsetSingleRowMode() doesn't work when getting batch results. The function is documented as: " int PQsetSingleRowMode(PGconn *conn); This function can only be called immediately after PQsendQuery or one of its sibling functions, before any other operation on the connect

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-09 Thread Daniel Verite
Vaishnavi Prabakaran wrote: >if (PQbatchStatus(st->con) != PQBATCH_MODE_ON) > >But, it is better to use if (PQbatchStatus(st->con) == > PQBATCH_MODE_OFF) for this verification. Reason is there is one more state > in batch mode - PQBATCH_MODE_ABORTED. So, if the batch mode is in ab

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-07 Thread Daniel Verite
Vaishnavi Prabakaran wrote: > Yes, I have created a new patch entry into the commitfest 2017-03 and > attached the latest patch with this e-mail. Please find attached a companion patch implementing the batch API in pgbench, exposed as \beginbatch and \endbatch meta-commands (without docum

Re: [HACKERS] One-shot expanded output in psql using \gx

2017-03-07 Thread Daniel Verite
Christoph Berg wrote: > Both fixed, thanks for the review! Version 3 attached. It looks good to me. Stephen Frost is also reviewer on the patch, so I'm moving the status back to "Needs review" at https://commitfest.postgresql.org/13/973/ and let him proceed. Best regards, -- Daniel Vé

Re: [HACKERS] One-shot expanded output in psql using \gx

2017-03-03 Thread Daniel Verite
Christoph Berg wrote: > The new version tests \g and \gx with a new query, and > re-running it on the last query buffer. Thanks, here's a review: The patch compiles and works as expected. The code follows the same pattern as other one-shot command modifiers, setting a flag in the global

Re: [HACKERS] \if, \elseif, \else, \endif (was Re: PSQL commands: \quit_if, \quit_unless)

2017-02-23 Thread Daniel Verite
Tom Lane wrote: > Ah, I see why *that* wants to know about it ... I think. I suppose you're > arguing that variable expansion shouldn't be able to insert, say, an \else > in a non-active branch? Maybe, but if it can insert an \else in an active > branch, then why not non-active too? See

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-04 Thread Daniel Verite
Corey Huinker wrote: [about Ctrl-C] > That does seem to be the consensus desired behavior. I'm just not sure > where to handle that. The var "cancel_pressed" shows up in a lot of places. > Advice? Probably you don't need to care about cancel_pressed, and the /if stack could be unwound at

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-02-03 Thread Daniel Verite
Corey Huinker wrote: > I meant in the specific psql-context, does it do anything other > than (attempt to) terminate sent-but-not-received SQL queries? It cancels the current edit in the query buffer, including the case when it spans multiple lines. If we add the functionality that Ctrl+C

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-02-02 Thread Daniel Verite
Peter Eisentraut wrote: > You really have (at least) three options here: > > 1. Rename nothing > 2. Rename directory only > 3. Rename everything I vote for 1) as I believe the renaming will create more confusion than it's worth, not even considering the renaming of functions and views.

Re: [HACKERS] Improvements in psql hooks for variables

2017-02-01 Thread Daniel Verite
Tom Lane wrote: > I updated the documentation as well. I think this is committable if > there are not objections. That works for me. I tested and read it and did not find anything unexpected or worth objecting. "\unset var" acting as "\set var off" makes sense considering that its opposi

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Daniel Verite
I wrote: > This would allow the hook to distinguish between initialization and > unsetting, which in turn will allow it to deny the \unset in the > cases when it doesn't make any sense conceptually (like AUTOCOMMIT). I notice that in the commited patch, you added the ability for DeleteVar

Re: [HACKERS] One-shot expanded output in psql using \G

2017-01-31 Thread Daniel Verite
Stephen Frost wrote: > That's not how '\dx' works, as I pointed out, so I don't see having the > second character being 'x' to imply "\x mode" makes sense. \gx means "like \g but output with expanded display" It turns out that it's semantically close to "\g with \x" so I refered to it lik

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Daniel Verite
Tom Lane wrote: > Moreover, the committed patch is inconsistent in that it forbids > only one of the above. Why is it okay to treat unset as "off", > but not okay to treat the default empty-string value as "on"? Treating unset (NULL in the value) as "off" comes from the fact that except

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-30 Thread Daniel Verite
Tom Lane wrote: > Evidently, this test script is relying on the preceding behavior that > setting a bool variable to an empty string was equivalent to setting > it to "true". If it's just that script I would be okay with saying > "well, it's a bug in that script" ... but I'm a bit worried

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-30 Thread Daniel Verite
Tom Lane wrote: > Also, if you want to argue that allowing it to change intra- > transaction is too confusing, why would we only forbid this direction > of change and not both directions? The thread "Surprising behaviour of \set AUTOCOMMIT ON" at: https://www.postgresql.org/message-id/CAH

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-30 Thread Daniel Verite
Corey Huinker wrote: > > \if ERROR > > \echo X > > \else > > \echo Y > > \endif > > > > having both X & Y printed and error reported on else and endif. I think > > that an expression error should just put the stuff in ignore state. > > > > Not just false, but ignore the wh

Re: [HACKERS] One-shot expanded output in psql using \G

2017-01-30 Thread Daniel Verite
Christoph Berg wrote: > But do we really want to choose > something different just because MySQL is using it? That's not what I meant. If mysql wasn't using \G I'd still suggest the name \gx because: - it means the functionality of \g combined with \x so semantically it makes sense. - t

Re: [HACKERS] One-shot expanded output in psql using \G

2017-01-28 Thread Daniel Verite
Christoph Berg wrote: > A workaround is to submit queries using "\x\g\x", but that's ugly, > clutters the output with toggle messages, and will forget that "\x > auto" was set. > > Mysql's CLI client is using \G for this purpose, and adding the very > same functionality to psql fits nicel

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-26 Thread Daniel Verite
Corey Huinker wrote: > Revised patch A comment about control flow and variables: in branches that are not taken, variables are expanded nonetheless, in a way that can be surprising. Case in point: \set var 'ab''cd' -- select :var; \if false select :var ; \else select 1; \endif The

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-24 Thread Daniel Verite
Corey Huinker wrote: > > > > 1: unrecognized value "whatever" for "\if"; assuming "on" > > > > I do not think that the script should continue with such an assumption. > > > > I agree, and this means we can't use ParseVariableBool() as-is The patch at https://commitfest.postgresql.org/12/

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-24 Thread Daniel Verite
I just wrote: > - add enum-style suggestions on invalid input for \pset x, \pset pager, > and \set of ECHO, ECHO_HIDDEN, ON_ERROR_ROLLBACK, COMP_KEYWORD_CASE, > HISTCONTROL, VERBOSITY, SHOW_CONTEXT, \x, \pager There's no such thing as \pager, I meant to write: \pset x, \pset pager, and \

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-24 Thread Daniel Verite
Tom Lane wrote: > I took a quick look through this. It seems to be going in generally > the right direction, but here's a couple of thoughts: Here's an update with these changes: per Tom's suggestions upthread: - change ParseVariableBool() signature to return validity as bool. - remov

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-24 Thread Daniel Verite
Tom Lane wrote: > However, it only really works if psql never overwrites the values > after startup, whereas I believe all of these are overwritten by > a \c command. Yes, there are reset to reflect the properties of the new connection. > So maybe it's more user-friendly to make these va

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-20 Thread Daniel Verite
Ashutosh Sharma wrote: > postgres=# \echo :ENCODING > UTF8 > postgres=# \set ENCODING xyz > postgres=# \echo :ENCODING > xyz > > I think currently we are not even showing what are the different valid > encoding names to the end users like we show it for other built-in > variables > VERBOS

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-16 Thread Daniel Verite
Tom Lane wrote: > "Daniel Verite" writes: > > [ psql-var-hooks-v6.patch ] > > I took a quick look through this. It seems to be going in generally > the right direction, but here's a couple of thoughts: Thanks for looking into this! > I'm incl

Re: [HACKERS] sequence data type

2017-01-13 Thread Daniel Verite
Peter Eisentraut wrote: > So in order to correctly answer the question, is the sequence about to > run out, you need to look not only at > the sequence but also any columns it is associated with. check_postgres > figures this out, but it's complicated and slow, and not easy to do > manual

Re: [HACKERS] sequence data type

2017-01-13 Thread Daniel Verite
Michael Paquier wrote: > Hm. Is symmetry an important properly for sequences? It seems to me > that if we map with the data types we had better use the MIN values > instead for consistency. So the concept of this patch is rather weird > and would introduce an exception with the rest of the

Re: [HACKERS] sequence data type

2017-01-10 Thread Daniel Verite
Peter Eisentraut wrote: > This could probably be sorted out somehow, but I don't want > to be too lax now and cause problems for later features. There is a > similar case, namely changing the return type of a function, which we > also prohibit. Consider the case of a table with a SERIAL

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2017-01-10 Thread Daniel Verite
Alvaro Herrera wrote: > With that, pushed and I hope this is closed for good. Great! I understand the fix couldn't be backpatched because we are not allowed to change the StringInfo struct in existing releases. But it occurs to me that the new "long_ok" flag might not be necessary after a

[HACKERS] Off-by-one oddity in minval for decreasing sequences

2017-01-06 Thread Daniel Verite
Hi, When testing the patch at https://commitfest.postgresql.org/12/768/ ("sequence data type" by Peter E.), I notice that there's a preexisting oddity in the fact that sequences created with a negative increment in current releases initialize the minval to -(2^63)+1 instead of -2^63, the actual l

Re: [HACKERS] Improvements in psql hooks for variables

2016-12-23 Thread Daniel Verite
Rahila Syed wrote: > Kindly consider following comments, Sorry for taking so long to post an update. > There should not be an option to the caller to not follow the behaviour of > setting valid to either true/false. OK, fixed. > In following examples, incorrect error message is begin d

Re: [HACKERS] [GENERAL] Select works only when connected from login postgres

2016-12-07 Thread Daniel Verite
Tom Lane wrote: > BTW, I realized while testing this that there's still one gap in our > understanding of what went wrong for you: cases like "SELECT 'hello'" > should not have tried to use the pager, because that would've produced > less than a screenful of data At some point emacs was m

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-12-05 Thread Daniel Verite
Alvaro Herrera wrote: > I have now pushed this to 9.5, 9.6 and master. It could be backpatched > to 9.4 with ease (just a small change in heap_form_tuple); anything > further back would require much more effort. > > I used a 32-bit limit using sizeof(int32). I tested and all the > menti

Re: [HACKERS] pgbench - allow backslash continuations in \set expressions

2016-12-01 Thread Daniel Verite
Fabien COELHO wrote: > - if not, are possible corner case backward incompatibilities introduced >by such feature ok? In psql, if backslash followed by [CR]LF is interpreted as a continuation symbol, commands like these seem problematic on Windows since backslash is the directory sepa

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-11-29 Thread Daniel Verite
Alvaro Herrera wrote: > But I realized that doing it this way is simple enough; > and furthermore, in any platforms where int is 8 bytes (ILP64), this > would automatically allow for 63-bit-sized stringinfos On such platforms there's the next problem that we can't send COPY rows through t

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-11-28 Thread Daniel Verite
Alvaro Herrera wrote: > I propose to rename allow_long to huge_ok. "Huge" is the terminology > used by palloc anyway. I'd keep makeLongStringInfo() and > initLongStringInfo() though as interface, because using Huge instead of > Long there looks strange. Not wedded to that, though (parti

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-23 Thread Daniel Verite
I wrote: > So I went through the psql commands which don't fail on parse errors > for booleans > [...] Here's a v5 patch implementing the suggestions mentioned upthread: all meta-commands calling ParseVariableBool() now fail when the boolean argument can't be parsed successfully. Also include

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-22 Thread Daniel Verite
Stephen Frost wrote: > That certainly doesn't feel right. I'm thinking that if we're going to > throw an error back to the user about a value being invalid then we > shouldn't change the current value. > > My initial thought was that perhaps we should pass the current value to > ParseVar

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-21 Thread Daniel Verite
Stephen Frost wrote: > Are you working to make those changes? I'd rather we make the changes > to this code once rather than push what you have now only to turn around > and change it significantly again. So I went through the psql commands which don't fail on parse errors for booleans.

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-21 Thread Daniel Verite
Stephen Frost wrote: > Are you working to make those changes? I'd rather we make the changes > to this code once rather than push what you have now only to turn around > and change it significantly again. If, as a maintainer, you prefer this together in one patch, I'm fine with that. So

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-21 Thread Daniel Verite
Stephen Frost wrote: > Just fyi, there seems to be some issues with this patch because setting > my PROMPT1 and PROMPT2 variables result in rather odd behavior. Good catch! The issue is that the patch broke the assumption of prompt hooks that their argument points to a long-lived buffer.

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-21 Thread Daniel Verite
Tom Lane wrote: > Stephen Frost writes: > > In reviewing this patch, I also noticed that it's set up to assume a > > 'true' result when a variable can't be parsed by ParseVariableBool. > > I suppose that's meant to be backwards-compatible with the current > behavior: > > regression=# \t

Re: [HACKERS] proposal: psql \setfileref

2016-11-15 Thread Daniel Verite
Corey Huinker wrote: > I am not asking for this feature now, but do you see any barriers to later > adding x/xq/xb/xbq equivalents for executing a shell command? For text contents, we already have this (pasted right from the doc): testdb=> \set content `cat my_file.txt` testdb=> INSERT I

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-15 Thread Daniel Verite
Hi, I'm attaching v3 of the patch with error reporting for FETCH_COUNT as mentioned upthread, and rebased on the most recent master. > I was suggesting change on the lines of extending generic_boolean_hook to > include enum(part in enum_hooks which calls ParseVariableBool) and > integer parsing

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-10 Thread Daniel Verite
Rahila Syed wrote: > I have applied this patch on latest HEAD and have done basic testing which > works fine. Thanks for reviewing this patch! > >if (current->assign_hook) > >- (*current->assign_hook) (current->value); > >- return true; > >+

Re: [HACKERS] [PATCH] Better logging of COPY queries if log_statement='all'

2016-10-17 Thread Daniel Verite
Aleksander Alekseev wrote: > According to my colleagues it would be very nice to have this feature. > For instance, if you are trying to optimize PostgreSQL for application > that uses COPY and you don't have access to or something like this. > It could also be useful in some other cases.

Re: [HACKERS] proposal: psql \setfileref

2016-10-10 Thread Daniel Verite
Gilles Darold wrote: >postgres=# \setfileref b /dev/random >postgres=# insert into test (:b); > > Process need to be killed using SIGTERM. This can be fixed by setting sigint_interrupt_enabled to true before operating on the file. Then ctrl-C would be able to cancel the command.

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2016-10-03 Thread Daniel Verite
Craig Ringer wrote: > I think it's mostly of interest to app authors and driver developers > and that's what it's aimed at. pg_restore could benefit a lot too. Wouldn't pgbench benefit from it? It was mentioned some time ago [1], in relationship to the \into construct, how client-server

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-09-30 Thread Daniel Verite
Tomas Vondra wrote: > A few minor comments regarding the patch: > > 1) CopyStartSend seems pretty pointless - It only has one function call > in it, and is called on exactly one place (and all other places simply > call allowLongStringInfo directly). I'd get rid of this function and >

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2016-09-28 Thread Daniel Verite
Tomas Vondra wrote: > 4) HandleParallelMessage needs a tweak, as it uses msg->len in a log > message, but with '%d' and not '%ld' (as needed after changing the type > to Size). This could be %zu for the Size type. It's supported by elog(). However, it occurs to me that if we don't clai

Re: [HACKERS] [PATCH] get_home_path: use HOME

2016-09-22 Thread Daniel Verite
Tom Lane wrote: > If we take this patch, what's to stop someone from complaining that we > broke *their* badly-designed system that abuses the HOME variable? POSIX warns against doing that, listing HOME in the variables that should be left to their intended usage: http://pubs.opengroup.or

Re: [HACKERS] Improvements in psql hooks for variables

2016-09-20 Thread Daniel Verite
Ashutosh Bapat wrote: > You may want to add this to the November commitfest > https://commitfest.postgresql.org/11/. Done. It's at https://commitfest.postgresql.org/11/799/ Best regards, -- Daniel Vérité PostgreSQL-powered mailer: http://www.manitou-mail.org Twitter: @DanielVerite --

Re: [HACKERS] Improvements in psql hooks for variables

2016-09-19 Thread Daniel Verite
Rahila Syed wrote: > I am beginning to review this patch. Initial comment. I got following > compilation error when I applied the patch on latest sources and run make. Sorry about that, I forgot to make clean, here's an updated patch. Best regards, -- Daniel Vérité PostgreSQL-powered m

[HACKERS] Improvements in psql hooks for variables

2016-09-16 Thread Daniel Verite
Hi, Following the discussion on forbidding an AUTOCOMMIT off->on switch mid-transaction [1], attached is a patch that let the hooks return a boolean indicating whether a change is allowed. Using the hooks, bogus assignments to built-in variables can be dealt with more strictly. For example, pre

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-15 Thread Daniel Verite
Tom Lane wrote: > I think changing the hook API is a pretty reasonable thing to do here > (though I'd make it its own patch and then add the autocommit change > on top). When was the last time you actually wanted to set VERBOSITY > to "fubar"? It looks easy to make the hooks return a boo

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-08 Thread Daniel Verite
Rahila Syed wrote: > However, including the check here will require returning the status > of command from this hook. i.e if we throw error inside this > hook we will need to return false indicating the value has not changed. Looking at the other variables hooks, they already emit errors

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2016-09-06 Thread Daniel Verite
Craig Ringer wrote: > Updated patch attached. Please find attached a couple fixes for typos I've came across in the doc part. Also it appears that PQqueriesInBatch() doesn't work as documented. It says: "Returns the number of queries still in the queue for this batch" but in fact it's i

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-03 Thread Daniel Verite
Rushabh Lathia wrote: > It might happen that SetVariable() can be called from other places in > future, > so its good to have all the set variable related checks inside > SetVariable(). There's already another way to skip the \set check: select 'on' as "AUTOCOMMIT" \gset But there's a

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-03 Thread Daniel Verite
Rahila Syed wrote: > >Above error coming because in below code block change, you don't have > check for > >command (you should check opt0 for AUTOCOMMIT command) > Corrected in the attached. There are other values than ON: true/yes and theirs abbreviations, the value 1, and anything that

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-29 Thread Daniel Verite
Joshua D. Drake wrote: > You log in, see that all the space and you find that you are using a > ton of disk space. You look around for anything you can delete. You > find a directory called pg_xlog, it says log, junior ignorant, don't > want to be a sysadmin 101 says, "delete logs". Yes,

Re: [HACKERS] Renaming of pg_xlog and pg_clog

2016-08-29 Thread Daniel Verite
Craig Ringer wrote: > People won't see a README in amongst 5000 xlog segments while > freaking out about the sever being down. Maybe they're more likely to google "pg_xlog". BTW, renaming it will not help with respect to that, as there's a pretty good corpus on knowledge linked to that pa

Re: [HACKERS] sslmode=require fallback

2016-07-20 Thread Daniel Verite
Magnus Hagander wrote: > > I don't understand why you want to change the default. Is it for > > performance? Has it been measured? > > > > > Yes. I've run into it multiple times, but I haven't specifically measured > it. But I've had more than one situation where turning it off has > com

Re: [HACKERS] \crosstabview fixes

2016-05-06 Thread Daniel Verite
Tom Lane wrote: > Pushed, thanks. > BTW, I see we've been spelling your name with an insufficient number > of accents in the commit logs and release notes. Can't do much about > the logs, but will fix the release notes. I use myself the nonaccented version of my name in "From" headers, a

Re: [HACKERS] \crosstabview fixes

2016-05-06 Thread Daniel Verite
Tom Lane wrote: > > "Daniel Verite" writes: > >> To avoid the confusion between "2:4" and "2":"4" or 2:4, > >> and the ambiguity with a possibly existing "2:4" column, > >> maybe we should abandon this sy

Re: [HACKERS] Rename max_parallel_degree?

2016-04-27 Thread Daniel Verite
Robert Haas wrote: > Of course, we could make this value 1-based rather than 0-based, as > Peter Geoghegan suggested a while back. But as I think I said at the > time, I think that's more misleading than helpful. The leader > participates in the parallel plan, but typically does far less

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Daniel Verite
Tom Lane wrote: > I wrote: > > My feeling is that what we should do is undo the change to use OT_SQLID, > > and in indexOfColumn() perform a downcasing/dequoting conversion that > > duplicates what OT_SQLID does in psqlscanslash.l. > > Here's an updated patch that does it that way, and al

Re: [HACKERS] \crosstabview fixes

2016-04-14 Thread Daniel Verite
Christoph Berg wrote: > If there's no way out, what about changing it the other way, i.e. > breaking the case where the column is named by a number? That seems > much less of a problem in practice. I don't think it would be acceptable. But there's still the option of keeping the dedicated

  1   2   >