Dynamic creation of list partitions in highly concurrent write environment

2023-06-01 Thread Jim Vanns
Hello everyone. We are attempting to evaluate list partitioning over hash partitioning (which we currently use) to better facilitate dropping tables that distinctly model devices we wish to purge. We don't want to use a DML statement for cleanup since these tables can contain billions of rows othe

Re: Hash Index on Partitioned Table

2023-06-01 Thread Marc Millas
Hi Peter, in postgres 13, create index should be, by default, parallelized. so albeit for specific values of the parallelization parameters in postgresql.conf, your machine should use more than one core while creating the indexes. also you can set the maintenance_workmem parameter to the max for s

RE: EXTERNAL: Re: Question - Does PostgreSQL have an Evaluation Assurance Level?

2023-06-01 Thread Mayer, Nicholas J
Hi Laurenz, Thanks for your reply but we are actually all set with this. We found out that while PostgreSQL does not have EAL, the 'Crunchy Data' does have EAL of 2. Please feel free to close/discontinue this question and discussion if you like. Thanks, Nick -Original Message- From:

RE: EXTERNAL: Re: Question - Does PostgreSQL have an Evaluation Assurance Level?

2023-06-01 Thread Mayer, Nicholas J
Hi Laurenz, Thanks for this information. That is correct, we are just ticking off the checkboxes at the moment but I appreciate your feedback. Thanks again, Nick -Original Message- From: Laurenz Albe Sent: Wednesday, May 31, 2023 4:31 PM To: Mayer, Nicholas J (US) ; pgsql-general@l

Re: Help needed to understand query planner regression with incremental sort

2023-06-01 Thread Henrik Peinar (nodeSWAT.com)
Thank you for taking the time to look this through. After reading your answer it obviously makes perfect sense, I was just thrown off by a fact that when it started happening, it happened on every query execution (for the same test query that I used). But I failed to think about trying different s

Re: Dynamic creation of list partitions in highly concurrent write environment

2023-06-01 Thread Tom Lane
Jim Vanns writes: > Here's the general approach we have, without code and with hypothetical > model, since I'm writing this hastily; > table devices; <-- After trigger on here > table routes; <-- To create physical partition for this logical table > table route_for_device_N; <-- The dynamically c

Re: Dynamic creation of list partitions in highly concurrent write environment

2023-06-01 Thread Jim Vanns
Yeah, that's a known concern - hence the evaluation as part of the list vs. hash comparison ;) Jim On Thu, 1 Jun 2023 at 14:32, Tom Lane wrote: > Jim Vanns writes: > > Here's the general approach we have, without code and with hypothetical > > model, since I'm writing this hastily; > > > tabl

Re: speed up full table scan using psql

2023-06-01 Thread Adrian Klaver
On 5/31/23 22:51, Lian Jiang wrote: The whole command is: psql %(pg_uri)s -c %(sql)s | %(sed)s | %(pv)s | %(split)s) 2>&1 | %(tr)s where: sql is "copy (select row_to_json(x_tmp_uniq) from public.mytable x_tmp_uniq) to stdout" sed, pv, split, tr together format and split the stdout into jsonl

[Beginner Question]A question about yacc & lex

2023-06-01 Thread Wen Yi
Hi team, now I'm learning the yacc & lex to understand the principle of the postgres's parser. And I write a test program as this: /*     array.l    Array program     Wen Yi */ %option noyywrap %option noinput %{ #include