Hey Tom
On Mon, Jun 22, 2020 at 10:13 PM Tom Lane wrote:
> Big -1 on that. COPY is not for general-purpose data transformation.
> The more unrelated features we load onto it, the slower it will get,
> and probably also the more buggy and unmaintainable.
what new format handling takes to add r
On Sun, Jan 20, 2019 at 2:24 AM Tomas Vondra
wrote:
>
> Pushed, thanks for the patch.
>
> cheers
>
>
>
Thank you
regards
Surafel
On Mon, Jan 21, 2019 at 6:22 PM Tomas Vondra
wrote:
>
> I think the condition can be just
>
> if (contain_volatile_functions(cstate->whereClause)) { ... }
>
>
>
yes it can be.
regards
Surafel
On Tue, Jan 22, 2019 at 3:35 PM David Rowley
wrote:
> On Sat, 19 Jan 2019 at 01:01, Surafel Temesgen
> wrote:
> > if you specified --inserts option you already specified the number of
> rows per statement which is 1 .
> > if more than one rows per statement needed it mu
On Wed, Jan 9, 2019 at 8:18 PM Tomas Vondra
wrote:
>
> See the attached patch, which recomputes the count regularly. I don't
> claim the patch is committable or that it has no other bugs, but
> hopefully it shows what I meant.
>
>
I got only one issue it is not work well with cursor
postgres=#
On Mon, Jan 28, 2019 at 1:28 AM Tomas Vondra
wrote:
>
>
> OK. Does that mean you agree the incremental approach is reasonable?
>
there are no noticeable performance difference but i love previous
approach more regarding cursor operation it fetch tuple forward and
backward from tuplestore only bu
here is a rebased version of previous patch with planner
comment included
regards
Surafel
On Wed, Jan 30, 2019 at 9:07 AM Surafel Temesgen
wrote:
>
>
> On Mon, Jan 28, 2019 at 1:28 AM Tomas Vondra
> wrote:
>
>>
>>
>> OK. Does that mean you agree the i
On Sun, Feb 3, 2019 at 11:00 AM Fabien COELHO wrote:
>
> Hello David,
>
> >> I do not understand why dump_inserts declaration has left the "flags for
> >> options" section.
> >
> > I moved that because it's no longer just a flag. It now stores an int
> value.
>
> Hmmm. Indeed, all th "int"s of th
On Sat, Feb 2, 2019 at 11:26 AM Fabien COELHO wrote:
>
> Hello David,
>
> > Wondering if you have anything else here? I'm happy for the v13
> > version to be marked as ready for committer.
>
> I still have a few comments.
>
> Patch applies cleanly, compiles, global & local make check are ok.
>
>
On Mon, Feb 4, 2019 at 8:29 AM Michael Paquier wrote:
>
> This patch needs a rebase because of conflicts done recently for
> pluggable storage, so moved to next CF, waiting on author.
> --
>
Thank you . rebased against current master
regards
Surafel
diff --git a/doc/src/sgml/ref/select.sgml b/d
On Mon, Feb 11, 2019 at 10:20 AM David Rowley
wrote:
> Reviewing pg_dump-rows-per-insert-option-v14.
>
Also, maybe one for Fabien (because he seems keen on keeping the
> --rows-per-insert validation code)
>
> strtol() returns a long. dump_inserts is an int, so on machines where
> sizeof(long) ==
On Sat, Feb 2, 2019 at 11:26 AM Fabien COELHO wrote:
> There is a test, that is good! Charater "." should be backslashed in the
> regexpr. I'd consider also introducing limit cases: empty table, empty
> columns by creating corresponding tables and using -t repeatedly
I see that there are alrea
On Mon, Feb 4, 2019 at 9:06 AM Michael Paquier wrote:
> On Wed, Dec 19, 2018 at 02:48:14PM +0300, Surafel Temesgen wrote:
> > Thank you for informing, attach is rebased patch against current
> > master
>
> copy.c conflicts on HEAD, please rebase. I am moving the patch to
On Sat, Feb 16, 2019 at 8:24 AM Andres Freund wrote:
> Hi,
>
> On 2018-08-23 17:11:04 +0300, Surafel Temesgen wrote:
> > COPY ... WITH ON CONFLICT LOG maximum_error, LOG FILE NAME '…';
>
> This doesn't seem to address Robert's point that a log file requ
On Tue, Feb 19, 2019 at 3:47 PM Andres Freund wrote:
>
>
> Err, what? Again, that requires super user permissions (in contrast to
> copy from/to stdin/out). Backends run as the user postgres runs under
>
okay i see it now and modified the patch similarly
regards
Surafel
diff --git a/doc/src/sg
On Sun, Feb 10, 2019 at 2:22 AM Tomas Vondra
wrote:
>
> I'm not sure I understand - are you saying every time the user does a
> FETCH, we have to run the outer plan from scratch? I don't see why would
> that be necessary? And if it is, how come there's no noticeable
> performance difference?
>
>
On Thu, Jan 31, 2019 at 10:22 PM Carter Thaxton
wrote:
>
>
> Here's a rebased patch that works with the latest master branch. Very
> straightforward.
>
You forget to resubmit it to the next commitfest and the error message on
incorrect
where clause specification is still the same
regards
Sur
On Tue, Dec 4, 2018 at 12:44 PM Alvaro Herrera
wrote:
> After reading this thread, I think I like WHERE better than FILTER.
> Tally:
>
> WHERE: Adam Berlin, Lim Myungkyu, Dean Rasheed, yours truly
> FILTER: Tomas Vondra, Surafel Temesgen
>
>
>
accepting tally result
On Wed, Dec 12, 2018 at 9:28 PM Tomas Vondra
wrote:
>
> Can you also update the docs to mention that the functions called from
> the WHERE clause does not see effects of the COPY itself?
>
>
*Of course, i also add same comment to insertion method selection*
*regards *
*Surafel*
diff --git a/d
On Fri, Nov 30, 2018 at 7:16 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> Unfortunately, patch needs to be rebased, could you please post an updated
> version?
>
Thank you for informing, Here is an updated patch against current master
Regards
Surafel
diff --git a/doc/src/sgml/ref/pg_dump.s
On Thu, Nov 29, 2018 at 3:15 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> Unfortunately, the patch conflict-handling-onCopy-from-v2.patch has some
> conflicts now, could you rebase it?
>
Thank you for informing, attach is rebased patch against current master
Regards
Surafel
diff --git a
Hi,
Currently we can skip header line on COPY FROM but having the ability to
skip and stop copying at any line can use to divide long copy operation and
enable to copy a subset of the file and skipping footer. Attach is a patch
for it
Regards
Surafel
diff --git a/doc/src/sgml/ref/copy.sgml b/do
On Tue, Dec 25, 2018 at 2:47 PM Fabien COELHO wrote:
Thank you for looking into it
>
> Hello Surafel,
>
> > Thank you for informing, Here is an updated patch against current master
>
> Patch applies cleanly, compiles, "make check" is okay, but given that the
> feature is not tested...
>
> Featur
On Fri, Dec 28, 2018 at 6:46 PM Fabien COELHO wrote:
>
> > At first i also try to do it like that but it seems the function will
> > became long and more complex to me
>
> Probably. But calling it with size 100 should result in the same behavior,
> so it is really just an extension of the preceed
On Tue, Jan 1, 2019 at 8:38 PM Tomas Vondra
wrote:
> >
> > The attached patch include all the comment given by Tomas and i
> > check sql standard about LIMIT and this feature
> >
>
> Unfortunately, it seems the "limit" regression tests fail for some
> reason - the output mismatches the ex
Hi,
Thank you for looking at it
On Mon, Dec 31, 2018 at 12:38 PM David Rowley
wrote:
> Just looking at the v5 patch, it seems not to handle 0 column tables
> correctly.
>
> For example:
>
> # create table t();
> # insert into t default values;
> # insert into t default values;
>
> $ pg_dump --tab
Hi
On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra
wrote:
> Hi,
>
> I've been looking at this patch today, and I think there's a couple of
> issues with the costing and execution. Consider a simple table with 1M
> rows:
>
> create table t as select i from generate_series(1,100) s(i);
>
> and
On Thu, Jan 3, 2019 at 4:51 PM Tomas Vondra
wrote:
>
> On 1/3/19 1:00 PM, Surafel Temesgen wrote:
> > Hi
> >
> > On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra
> > mailto:tomas.von...@2ndquadrant.com>>
> wrote:
>
> > The execution part o
On Tue, Jan 1, 2019 at 10:08 PM Tomas Vondra
wrote:
> The execution part of the patch seems to be working correctly, but I
> think there's an improvement - we don't need to execute the outer plan
> to completion before emitting the first row. For example, let's say the
> outer plan produces 1
On Thu, Jan 3, 2019 at 1:38 AM David Rowley
wrote:
> On Thu, 3 Jan 2019 at 01:50, Surafel Temesgen
> wrote:
> > On Mon, Dec 31, 2018 at 12:38 PM David Rowley <
> david.row...@2ndquadrant.com> wrote:
> >> Just looking at the v5 patch, it seems not to handle 0 column
Hi,
On Fri, Jan 4, 2019 at 5:37 PM Peter Eisentraut <
peter.eisentr...@2ndquadrant.com> wrote:
> It seems a bit fragile to me if I want to skip a footer and need to
> figure out the total line count, subtract one, and then oh, was it zero-
> or one-based.
>
>
But normally we don't say start copyin
On Fri, Jan 4, 2019 at 5:27 PM Tomas Vondra
wrote:
>
> What formula? All the math remains exactly the same, you just need to
> update the number of rows to return and track how many rows are already
> returned.
>
> I haven't tried doing that, but AFAICS you'd need to tweak how/when
> node->count
Hi,
On Sat, Jan 5, 2019 at 1:10 PM David Rowley
wrote:
> On Fri, 21 Dec 2018 at 02:02, Surafel Temesgen
> wrote:
> > Currently we can skip header line on COPY FROM but having the ability to
> skip and stop copying at any line can use to divide long copy operation and
> enabl
On Sun, Jan 6, 2019 at 5:51 PM Tomas Vondra
wrote:
>
> On 1/6/19 12:40 PM, Surafel Temesgen wrote:
> >
> >
> > On Fri, Jan 4, 2019 at 5:27 PM Tomas Vondra
> > mailto:tomas.von...@2ndquadrant.com>>
> wrote:
> >
> >
> > What formula
On Wed, Jan 9, 2019 at 5:38 PM Tomas Vondra
wrote:
>
> It's hard to say what exactly are you doing, because you haven't shared
> any code nor examples.
okay i attach in progress patch
regards
Surafel
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 4db8142afa..849
On Wed, Jan 2, 2019 at 6:19 PM Tomas Vondra
wrote:
> After looking at the "FETCH FIRST ... PERCENT" patch, I wonder if this
> patch should tweak estimates in some way. Currently, the cardinality
> estimate is the same as for plain LIMIT, using the requested number of
> rows. But let's say there a
On Tue, Jan 15, 2019 at 2:51 PM Tomas Vondra
wrote:
> What do you mean by "raw statistic"?
>
I mean without further calculation to consider other operation
>
> IMHO the best thing you can do is call estimate_num_groups() and combine
> that with the number of input rows. That shall benefit fro
On Fri, Jan 4, 2019 at 3:08 PM David Rowley
wrote:
> On Mon, 31 Dec 2018 at 18:58, Surafel Temesgen
> wrote:
> > On Fri, Dec 28, 2018 at 6:46 PM Fabien COELHO
> wrote:
> >> > At first i also try to do it like that but it seems the function will
> >>
On Fri, Jan 18, 2019 at 7:14 AM David Rowley
wrote:
> On Fri, 18 Jan 2019 at 01:15, Surafel Temesgen
> wrote:
> > The attache patch use your method mostly
>
> I disagree with the "mostly" part. As far as I can see, you took the
> idea and then made a series of
On Fri, Jan 18, 2019 at 2:29 PM David Rowley
wrote:
> On Fri, 18 Jan 2019 at 19:29, Surafel Temesgen
> wrote:
> > this happen because i don't disallow the usage of --inserts and
> --rows-per-insert
> > option together.it should be error out in those case.i correct it
Hi,
On Tue, Feb 4, 2020 at 4:25 PM Rémi Lapeyre wrote:
> This patch adds the possibility to use the "header" option when using COPY
> with
> the text format. A todo entry was opened for this and I updated the tests
> and
> the documentation.
>
> This was previously discussed at
> https://www.post
On Wed, Feb 5, 2020 at 4:19 PM Rémi Lapeyre wrote:
> >
> > FWIW there was more recent propose patch at
> https://www.postgresql.org/message-id/flat/caf1-j-0ptcwmeltswwgv2m70u26n4g33gpe1rckqqe6wvqd...@mail.gmail.com
> > and among feedback given is to adding header matching feature on to
> this.
>
Hi,
> > ERROR_LIMIT ' class="parameter">limit_number'
> >
> > I think this should be:
> >
> > ERROR_LIMIT limit_number
> >
> > (no single quote)
>
>
Thank you .Fixed
> More comments:
>
> - I think the document should stat that if limit_number = 0, all
> errors are immediately raised (be
On Mon, Feb 17, 2020 at 10:00 AM Tatsuo Ishii wrote:
> >> test=# copy t1 from '/tmp/a' with (error_limit 1);
> >> ERROR: duplicate key value violates unique constraint "t1_pkey"
> >> DETAIL: Key (i)=(2) already exists.
> >> CONTEXT: COPY t1, line 2: "2 2"
> >>
> >> So if the number of errors
On Mon, Mar 2, 2020 at 2:45 AM Rémi Lapeyre wrote:
>
> I created an entry for this patch in the new CommiFest but it seems that
> it is not finding it. Is there anything that I need to do?
>
>
Is is added on next open commit fest which is
https://commitfest.postgresql.org/28/ now
regards
Suraf
On Fri, Mar 6, 2020 at 11:30 AM asaba.takan...@fujitsu.com <
asaba.takan...@fujitsu.com> wrote:
> Hello Surafel,
>
> Sorry for my late reply.
>
> From: Surafel Temesgen
> >On Thu, Dec 12, 2019 at 7:51 AM mailto:asaba.takan...@fujitsu.com
> <mailto:asaba.takan...@
Hello,
The attached patch include the fix for all the comment given
regards
Surafel
On Mon, Apr 1, 2019 at 12:34 AM Andres Freund wrote:
> Hi,
>
> On 2019-03-29 12:04:50 +0300, Surafel Temesgen wrote:
>
> > + if (node->limi
On Wed, Feb 20, 2019 at 7:04 PM Andres Freund wrote:
>
>
> On February 20, 2019 6:05:53 AM PST, Andrew Dunstan <
> andrew.duns...@2ndquadrant.com> wrote:
> >
> >On 2/20/19 8:01 AM, Surafel Temesgen wrote:
> >>
> >>
> >> On Tue, Feb 19, 201
Hi Thomas
On Mon, Jul 1, 2019 at 1:31 PM Thomas Munro wrote:
> On Mon, Apr 8, 2019 at 8:26 PM Surafel Temesgen
> wrote:
> > agree
>
> Hi Surafel,
>
> A new Commitfest is starting. Can you please post a fresh rebase of this
> patch?
>
>
Thank you for informing.
Hello,
we use atoi for user argument processing in same place which return zero
for both invalid input and input value zero. In most case its ok because we
error out with appropriate error message for input zero but in same place
where we accept zero as valued input it case a problem by preceding
Hi,
Here are same review comment
- any numeric, string, date/time, network, or enum type,
+ any numeric, string, date/time, network, lsn, or enum type,
or arrays of these types
same as argument type
In the documentation it refereed as pg_lsn type rather than lsn alone
Hi Erik,
Thank you for looking at it.
Can you have a look?
>
>
It is because of the patch didn't handle that edge case.
attache patch contain a fix for it
regards
Surafel
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 06d611b64c..e83d309c5b 100644
--- a/doc/src/sg
Hi Alexey,
Thank you for looking at it
On Tue, Jul 2, 2019 at 7:57 PM Alexey Kondratov
wrote:
> On 28.06.2019 16:12, Alvaro Herrera wrote:
> >> On Wed, Feb 20, 2019 at 7:04 PM Andres Freund
> wrote:
> >>> Or even just return it as a row. CopyBoth is relatively widely
> supported
> >>> these day
Hi Thomas,
Thank you for informing me
Hi Surafel,
>
> There's a call to adjust_limit_rows_costs() hiding under
> contrib/postgres_fdw, so this fails check-world.
>
Fixed . I also include the review given by Ryan in attached patch
regards
Surafel
diff --git a/contrib/postgres_fdw/postgres_fdw.c b
Hi Ryan,
On Tue, Jul 9, 2019 at 1:27 AM Ryan Lambert wrote:
> Hi Surafel,
>
> The v5 patch [1] applies cleanly and passes make installcheck-world.
>
> My concern with this is its performance. As a user I would expect using
> this feature to enable queries to run faster than they would simply pul
Hi Takamichi Osumi,
On Tue, Jul 9, 2019
> I've rebased the previous patch to be applied
>
I don't test your patch fully yet but here are same comment.
There are same white space issue like here
- bool is_internal)
+ bool is_internal,
+ Oid existing_constraint_oid)
in a few place
+ // trigoid
Hi Alvaro,
On Wed, Jul 10, 2019 at 6:44 PM Alvaro Herrera
wrote:
> What's with the new tuplestore function for getting heap tuples? That
> looks really odd.
>
Previously I create new TTSOpsMinimalTuple type slots for every tuple
returned in order to fetch it from tuplestore because tuplestore
Hi
>
> Also, I would prefer having an option to ignore all errors, e.g. with
> option ERROR_LIMIT set to -1. Because it is rather difficult to estimate
> a number of future errors if you are playing with some badly structured
> data, while always setting it to 100500k looks ugly.
>
Here are the p
On Sun, Jul 14, 2019 at 7:40 PM Alvaro Herrera
wrote:
>
> error_limit being an integer, please don't use it as a boolean:
>
> if (cstate->error_limit)
>
...
>
> Add an explicit comparison to zero instead, for code readability.
> Also, since each error decrements the same variable, it becomes
Hi Ryan,
On Tue, Jul 9, 2019 at 4:13 PM Ryan Lambert wrote:
>
> "It is possible for FETCH FIRST N PERCENT to create poorly performing
> query plans when the N supplied exceeds 50 percent. In these cases query
> execution can take an order of magnitude longer to execute than simply
> returning th
hey,
i am reviewing this patch
On Thu, May 31, 2018 at 4:49 AM, Carter Thaxton
wrote:
>
> pg_dump --where '"foo:bar":created_at >= '2018-05-01'" dbname
>
it would be more sqlish if it specified like:
--table=foo --where ="bar created_at >= 2018-05-01"
and i don't like the idea of duplicating t
On Mon, Jul 2, 2018 at 9:27 PM, Robert Haas wrote:
>
> But you can specify multiple tables. You wouldn't want the same WHERE
> clause to apply to all of them.
>
> also with this new --where option you can specify multiple table using
wildcard and it
try to apply the same where clause to each tab
Hellow hackers,
A few commitfest ago there was same effort to add errors handling to COPY
FROM[1] and i see there that we already have infrastructure for supporting
handling of unique violation or exclusion constraint violation error and I
think it is independently useful too. Attached is a patch
On Fri, Jul 31, 2020 at 1:38 AM Daniel Gustafsson wrote:
>
> > $ pg_dump -d cary --where="test1:a3 = ( select max(aa1) from test2 )" >
> testdump2
> > $ pg_dump: error: processing of table "public.test1" failed
> >
> > both test1 and test2 exist in the database and the same subquery works
> und
On Thu, Mar 5, 2020 at 4:17 AM Krasiyan Andreev wrote:
> I have currently suspended development of this patch, based on it's
> review,
> but I will continue development of the other Oliver Ford's work about
> adding support of respect/ignore nulls
> for lag(),lead(),first_value(),last_value() and
Hi Michael
On Thu, Sep 24, 2020 at 6:58 AM Michael Paquier wrote:
> On Mon, Aug 10, 2020 at 01:23:44PM +0300, Surafel Temesgen wrote:
> > I also Implement PERCENT WITH TIES option. patch is attached
> > i don't start a new tread because the patches share common code
>
&g
Hello,
Sometimes I have to maintain two similar database and I have to update one
from the other and notice having the option to add ON CONFLICT DO NOTHING
clause to INSERT command in the dump data will allows pg_restore to be done
with free of ignore error.
The attache patch add --on-conflect-do
On Tue, Jun 12, 2018 at 12:05 PM, Ideriha, Takeshi <
ideriha.take...@jp.fujitsu.com> wrote:
thank you for the review
> Hi,
> I feel like that on-conflict-do-nothing support is useful especially
> coupled with --data-only option.
> Only the difference of data can be restored.
>
> >The attache patch
On Sat, Jun 16, 2018 at 11:36 AM, Dilip Kumar wrote:
>
> @@ -172,6 +172,7 @@ typedef struct _dumpOptions
> char*outputSuperuser;
>
> int sequence_data; /* dump sequence data even in schema-only mode */
> + int do_nothing;
> } DumpOptions;
>
> The new structure member appears out of place
FETCH FIRST with PERCENT option is SQL standard that state limit count to
be specified in a percentage in addition to specify it in exact count and
listed as one of Major features simply not implemented yet in recent wiki
page [1].
I implemented it by executing the query plan twice. One without li
Thanks for looking at it
1. It sounded like you added the copy_max_error_limit GUC as part of this patch
to allow users to specify how many errors they want to swallow with this new
functionality. The GUC didn't seem to be defined and we saw no mention of it in
the code. My guess is this might
On Thu, Aug 16, 2018 at 5:34 PM, Andres Freund wrote:
>
> Won't that have rather massive issues with multiple evaluations of
> clauses in the query? Besides being really expensive?
>
The plan re-scane after first execution I can’t see issue for multiple
execution of a clause in this case
>
> I
Hello,
The attached patch add error handling for
Extra data
missing data
invalid oid
null oid and
row count mismatch
And the record that field on the above case write to the file with appended
error message in it and in case of unique violation or exclusion constraint
violation error the fail
On Tue, Aug 21, 2018 at 3:50 PM Andres Freund wrote:
>
> Imagine volatile functions being used. That can be problematic because
> of repeated side-effects, but also will lead to plainly wrong
> results. Consider what happens with your approach where somebody does
> something like WHERE value < ra
On Tue, Aug 28, 2018 at 7:33 PM Erik Rijkers wrote:
> ;
>
> TRAP: FailedAssertion("!(slot != ((void *)0))", File: "execTuples.c",
> Line: 42
The attache patch include a fix for the crash .can you check it again?
Regards
Surafel
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.
On Tue, Jul 3, 2018 at 1:11 AM, Carter Thaxton
wrote:
> The whole reason for the colon in the --where option is to indicate which
> table the WHERE clause should refer to, so that one can dump less than all
> of the rows.
> The --table option is totally different. It specifies which tables to
>
Hey ,
Here are a few comment.
+
Here's a typo: ="libpq-connect-falover-timeout"
+ {"failover_timeout", NULL, NULL, NULL,
+ "Failover Timeout", "", 10,
Word is separated by hyphen in internalPQconninfoOption lable as a
surrounding code
+If the value is random, the ho
hey
On 9/21/18, Mark Dilger wrote:
> Surafel, there are no regression tests that I can see in your patch. It
> would help if you added some, as then I could precisely what behavior you
> are expecting.
thank you for looking at it .the attach patch add regression tests
diff --git a/doc/src/sgml/
101 - 178 of 178 matches
Mail list logo
