har(now() at time zone 'Europe/Amsterdam', '-MM-DD
HH24:MI:SS.US0 TZH:TZM');
to_char
----
2024-01-11 16:24:52.8736860 +00:00
(1 row)
At least it seems to do what you need.
Regards,
Michael
>
> I'm dealing with a data virt
ah, and for completeness the simple python function I use for the test:
create or replace function reports.generic_query_python(_sql text)
RETURNS SETOF record
LANGUAGE 'plpythonu'
PARALLEL SAFE
COST 100
VOLATILE
ROWS 5000
AS $BODY$
return plpy.execute( _sql )
$BODY$;
-
Function Scan on generic_query_python foo (cost=0.25..50.25 rows=5000
width=40) (actual time=49.920..49.922 rows=17 loops=1)
Planning time: 0.029 ms
Execution time: 49.977 ms
(3 rows)
Michael Krüger schrieb am F
hink
that your backups taken are safe, until you see a corruption which has
been hidden for weeks.
--
Michael
signature.asc
Description: PGP signature
rewind, so you could stop the slave 2,
and run pg_rewind unconditionally to keep things simple.
--
Michael
signature.asc
Description: PGP signature
wn checkpoint,
while doing it on slave 2 may be optional, still safer to do.
--
Michael
signature.asc
Description: PGP signature
On Thu, Mar 08, 2018 at 10:48:29AM +0300, Yavuz Selim Sertoğlu wrote:
> If not set, could you add recovery.conf file
> recovery_target_timeline='latest'
> parameter?
> https://www.postgresql.org/docs/devel/static/recovery-target-settings.html
Yes, that's visibly
according to the
> wiki.
You should update and upgrade. 9.1 has fallen out of community support
1 year and a half ago, and 9.1.9 is utterly outdated.
--
Michael
signature.asc
Description: PGP signature
f List-Unsubscribe and you are good to go. I don't know
much how email clients parse that, but gmail actually shows that as a
small button you can click on if I recall correctly.
--
Michael
signature.asc
Description: PGP signature
oops through all the connections to finish
them. Now would we want to slow down the session shutdown just for
that? I am less sure particularly if there is lag between the remote
and the local servers.
--
Michael
signature.asc
Description: PGP signature
PQencryptPasswordConn and
PQencryptPassword are useful. Using psql's \password is a good habit to
have.
--
Michael
signature.asc
Description: PGP signature
ther things, in which case the upgrade
to a newer PostgreSQL version, if need be, will be taken care of by the
product itself. If you are using your own set of PostgreSQL instances,
then that's up to what you have at OS level.
--
Michael
signature.asc
Description: PGP signature
On Mon, Mar 26, 2018 at 12:46:38PM -0300, Alvaro Herrera wrote:
> Was this ever fixed?
Ugh. I have added a reminder on the open item page for v11 as an older
bug:
https://wiki.postgresql.org/wiki/PostgreSQL_11_Open_Items#Older_Bugs
--
Michael
signature.asc
Description: PGP signature
a strict answer before knowing what you are
looking for in terms of WAL archive redundancy.
You may need to use target_timeline = 'latest' in your recovery.conf
settings as well.
--
Michael
signature.asc
Description: PGP signature
o do the work I am mentioning above to get back to a clean state.
--
Michael
signature.asc
Description: PGP signature
that you forgot to drop on slave 1?
--
Michael
signature.asc
Description: PGP signature
a standby a higher
catchup window. Giving value to one or the other depends on the
context, and both are usable in completely different circumstances.
--
Michael
signature.asc
Description: PGP signature
n deleted, though this requires a
special set of skills and luck. Hiring a professional is recommended
for such operations.
--
Michael
signature.asc
Description: PGP signature
On Mon, Apr 16, 2018 at 6:58 PM, Ron wrote:
> We're upgrading from v8.4 to 9.6 on a new VM in a different DC. The dump
> file will be more than 1TB, and there's not enough disk space on the
> current system for the dump file.
>
> Thus, how can I send the pg_dump file directly to the new server w
o do its work as
the oldest XID in view is not updated, causing performance to go down,
and bloat to go up.
--
Michael
signature.asc
Description: PGP signature
x_wal_size is of course a soft limit, and it has been discussed a
couple of times that it would be nice to get that to a hard limit, but
it is really a hard problem to avoid the system to not slow down or even
stop its I/O if the hard is close by or reached..
--
Michael
signature.asc
Description: PGP signature
taining a bit
more WAL than necessary, but a client can request WAL data using a
replication slot which is at least the oldest position. Asking for a
newer position would cause the replication slot to just not retain
anymore the data past the point requested.
--
Michael
signature.asc
Description: PGP signature
As suggested, note in ToDo table, also maybe look at LISTEN and NOTIFY
and have a job runner process LISTENing (and cleaning up the queue,
or, marking an item as in progress if you've multiple workers) The
work queue table is to help maintain state...if noone is LISTENing
then the table acts as ba
unction as well. I recall that Amit Langote has
sent a patch which introduces a wrapper function on top of
find_all_inheritors, perhaps that would get into v12.
--
Michael
signature.asc
Description: PGP signature
ived, and then
allow the standby to recover the segment only after a delta has passed.
The can allow a more evenly distribution of segments.
--
Michael
signature.asc
Description: PGP signature
when a stop point happens or not. It
also has no idea of the configuration within recovery.conf, which is
loaded by the startup process.
> Yes, but thats far less simple than just setting restore_target_time .
It seems to me that archiving provides the control you are looking for.
--
Michael
even if an administrator disables
connections to it.
--
Michael
signature.asc
Description: PGP signature
ma? Are you using the native partitioning instead?
--
Michael
signature.asc
Description: PGP signature
On Wed, May 16, 2018 at 6:36 PM, Adrian Klaver
wrote:
> On 05/16/2018 03:19 PM, hmidi slim wrote:
>>
>> HI,
>>
>> I'm working on a microservice application and I avoid using triggers
>> because they will not be easy to maintain and need an experimented person in
>> database administration to manag
On Thu, May 17, 2018 at 08:31:48AM +0100, Paul Linehan wrote:
> I'm having problems compiling PostgreSQL.
On which platform and/or distribution are you trying the code
compilation?
--
Michael
signature.asc
Description: PGP signature
initdb is failing on Centos 7 with 10.4 because the install procedure does
not change the ownership of the /usr/local/pgsql directory to postgres.
Changing the ownership fixes the problem, but the install procedure should
do this.
--
Mike Nolan
On Sat, May 19, 2018 at 2:16 PM, Don Seiler wrote:
> On Sat, May 19, 2018 at 2:10 PM, Michael Nolan wrote:
>
>> initdb is failing on Centos 7 with 10.4 because the install procedure
>> does not change the ownership of the /usr/local/pgsql directory to
>> postgres.
>
On Sat, May 19, 2018 at 5:23 PM, Adrian Klaver
wrote:
> On 05/19/2018 03:16 PM, Michael Nolan wrote:
>
>>
>>
>> On Sat, May 19, 2018 at 2:16 PM, Don Seiler > d...@seiler.us>> wrote:
>>
>> On Sat, May 19, 2018 at 2:10 PM, Michael
Just call me blind!
--
Mike Nolan
On Tue, May 22, 2018 at 10:35:46PM -0700, David G. Johnston wrote:
> I would conclude that pg-basebackup is placing its output in stderr instead
> of stdout then...
The output of pg_basebackup's verbose mode goes to stderr (look for
example at the verbose flags in pg_basebackup.c).
ted on the primary may not
be able to replay on its standbys. Note however that you won't get
support for such configurations on the community lists, so just make
sure that all nodes in a cluster are on the same version and that it is
the latest one.
--
Michael
signature.asc
Description: PGP signature
Microsoft has bought GitHub for $7.5 billion, is this a threat to the open
source community?
--
Mike Nolan
On Mon, Jun 4, 2018 at 12:15 PM, Tom Lane wrote:
> Michael Nolan writes:
> > Microsoft has bought GitHub for $7.5 billion, is this a threat to the
> open
> > source community?
>
> A fair question, but one that seems entirely off-topic for the Postgres
> lists, sinc
y is to take automatic backups of what they have on those
places and keep a copy of them locally. This way, you have an exit door
if something happens to the place where the code is located. I do so
for all my stuff on github for example. And git makes that really easy
to do.
--
Michael
signature.asc
On Thu, Jun 07, 2018 at 11:57:06AM +0200, Jehan-Guillaume (ioguix) de Rorthais
wrote:
> How do you backup your projects issues on github? Using the API with some
> loops?
I personally don't care much about this part. As long as the code
survives..
--
Michael
signature.asc
Descr
hat could make sense to
use numeric as well here, which is not the greatest choice by the way as
that's an int64 internally, but that's more portable for any
(unlikely-to-happen) future changes.
Using the segment size value in pg_settings, you could also advance the
LSN worth a full s
say what you are especially looking for that psql's \copy cannot
do, but perhaps you have an interest in pg_bulkload? Here is a link to
the project:
https://github.com/ossc-db/pg_bulkload/
It has a couple of fancy features as well, like preventing failures of
rows if loading a large file, etc
le to recover up to the wanted recovery
point. These backups are typically much faster to backup and restore
than pg_dump for large deployments but can result as well in larger
backup sizes, so the speed of one method or the other is to evaluate
carefully first."
I am open to better suggestions of
Hi Pierre,
On Wed, Jun 20, 2018 at 08:06:31AM +, Pierre Timmermans wrote:
> Hi Michael
You should avoid top-posting on the Postgres lists, this is not the
usual style used by people around :)
> Thanks for the confirmation. Your rewording removes the confusion. I
> would maybe
Gmail allows '>' when you click on the dots to expand the mail you
> are replying to, but it messes up in justifying and formatting it.
Those products have good practices when it comes to break and redefine
what the concept behind emails is...
--
Michael
signature.asc
Description: PGP signature
; backup is considered a "hot backup". The rest is style to make that flow
> better.
Indeed. The section uses hot backups a lot.
What do all folks here think about the updated attached?
--
Michael
diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml
index 982776ca0a..af48a
pplies cleanly
anyway as this comes from the period where hot standbys have been
introduced. So that would not be a lot of work... Speaking of which,
it would be nice to be sure about the wording folks here would prefer
using before fixing anything ;p
--
Michael
signature.asc
Description: PGP signature
entation improvement, I think
that it would be welcome, say in the replication protocol page for the
command START_REPLICATION. This level of details is really internal,
but fresh ideas may bring more clarity in this area, and nowhere in the
docs are used references to either 0/0 or InvalidXLo
uestion I am pretty sure that you mean "B" instead of "A" as
"A" has died and is off the grid. The answer to that question would be
perhaps no, as if C could have replayed WAL ahead of B in which case you
could finish with a corrupted C instance if trying to re
','ClienteWrite') and state='idle'.
>
> Can anyone help me?
You are visibly looking for a WHERE clause defined with state IN
('idle', 'idle in transaction') when looking for connections waiting for
some activity to be generated by the application, which works also with
9.6.
--
Michael
signature.asc
Description: PGP signature
ore step 7. This makes the promoted standby update its
timeline number in the on-disk control file, which is used by pg_rewind
to check if a rewind needs to happen or not.
We see too many reports of such mistakes, I am going to propose a patch
on the -hackers mailing list to mention that in the documentation...
--
Michael
signature.asc
Description: PGP signature
is a soft size, and not a hard size, hence
depending on your worload you may see more WAL segments than what is set
in 16GB. The docs mention that, so no need to be surprised.
--
Michael
signature.asc
Description: PGP signature
d potentially a patch for a
new feature which makes pg_rewind trigger directly a checkpoint.
--
Michael
signature.asc
Description: PGP signature
n HEAD (look at
ControlFile_source, you would see that only the new TLI matters, and
that getTimelineHistory does not really need to know the contents of the
control file).
--
Michael
signature.asc
Description: PGP signature
uld
get a no-op with pg_rewind, and when restarting the old master witha
recovery.conf you would get a failure. If you stop the old master
so as at next startup it needs crash recovery to recover, then there is
indeed a risk of corrupted instance, but that would be the same problem
even if pg_rewind
ation against 10)
Well, pglogical has the advantage of reducing the downtime, which may
not matter depending on your application and you may be able to accept a
it of downtime, and pg_upgrade --link can be pretty quick at its job.
Test it before as --link is a no-return trip.
--
Michael
signature.asc
Description: PGP signature
o be bitten.
Yeah, I have not really come up with a nice idea yet, especially when
things sometimes move with custom files that some users have been
deploying, so I am not completely sure that we'd need to do something
anyway, nor that it is worth the trouble. One saner strategy may be to
split your
be to
take a base backup without WAL segments included and with a WAL archive
used by the base backup taken. A second is to use a physical
replication slot which guarantee the presence of the wanted segments.
--
Michael
signature.asc
Description: PGP signature
ING is created during a CREATE TABLESPACE if you
do that.
--
Michael
signature.asc
Description: PGP signature
th backends doing
read-only operations on a standby once it has reached a consistent
point?
--
Michael
signature.asc
Description: PGP signature
ent page issues would actually
show up when they should, and that those become reproducible so as we
can track down any rogue WAL record or inconsistent behavior.
--
Michael
signature.asc
Description: PGP signature
nstallations).
If there are folks willing to put enough effort in getting this to work,
it could work, assuming that a buildfarm animal is able to get down this
road. From what I can see on this thread we are not yet at that stage
though.
--
Michael
signature.asc
Description: PGP signature
which is where
the SHA2 interface for OpenSSL has been introduced. So likely we'd need
to look into that more deeply.. This has the strong smell of a bug. If
your system is new enough, you should have sha256() & co as system
functions, so you would see the failure as well? The reg
On Wed, Sep 05, 2018 at 01:19:39PM +, Alessandro Gherardi wrote:
> Hi Michael,I'm actually running postgres on Windows.
First you may want to avoid top-posting. This is not the style of the
community lists and this breaks the logic of a thread.
> I added code to fe-secure-opens
can be used when an instance is in recovery.
--
Michael
signature.asc
Description: PGP signature
that down to v10 where SCRAM has been introduced.
--
Michael
signature.asc
Description: PGP signature
On Tue, Sep 11, 2018 at 12:02:50PM +0900, Michael Paquier wrote:
> Hence, intrinsically, we are in contradiction with the upstream docs. I
> have worked on the problem with the patch, which works down to OpenSSL
> 0.9.8, and should fix your issue. This is based on what you sent
>
gged with OpenSSL correctly are two separate
issues. The second one also says that we are in the grey based on
OpenSSL docs, which worryies me. And EVP_DigestInit is used in pgcrypto
for ages, where I don't recall seeing reports about that.
--
Michael
signature.asc
Description: PGP signature
On Wed, Sep 12, 2018 at 07:24:24AM +0900, Michael Paquier wrote:
> Good point. Such things have bitten in the past. Okay, then let's do
> something about sha2_openssl.c only on HEAD for now then, which I am
> fine to finish wrapping.
I was looking at trying to commit this patch
omething similar to what happens on Linux with a
system-wide switch that Postgres knows nothing about. Perhaps that will
not be the case, but let's think about that once we know for sure.
--
Michael
signature.asc
Description: PGP signature
e trying to replay.
In this case a contrecord refers to a WAL record split across multiple
pages. The WAL reader is expecting one, and cannot find it. And that's
not normal. My bet is that something is wrong in your failover flow
which you think is right. It is hard to get that right.
--
by us in
> “postgresql.conf” in order to recycle the log files after 30 days.
If you use for example log_filename = 'postgresql-%d.log', then the
server uses one new file every day. This truncates the contents from
the last month automatically.
--
Michael
signature.asc
Description: PGP signature
On Fri, Sep 28, 2018 at 06:19:16AM -0700, Adrian Klaver wrote:
> If log_truncate_on_rotation = 'on', correct?
Yup, thanks for precising.
--
Michael
signature.asc
Description: PGP signature
imary use more CPU than necessary so you can
leverage the activity on a standby.
--
Michael
signature.asc
Description: PGP signature
y to ensure the
uniqueness of this file, as two backups could be started in parallel
with the same start location and the *same* file name. If an archive
command is not able to handle correctly the duplicates, you could bloat
pg_wal. And that's a real problem.
--
Michael
signature.asc
Description: PGP signature
rchive twice the same file, then the
archive command would continuously fail and prevent existing WAL
segments to be archived. Segments are continuously created, and pg_wal
grows in size.
--
Michael
signature.asc
Description: PGP signature
ry files having the same name gets larger
the more the window between two checkpoints is. That's a problem I
studied a couple of months back.
--
Michael
signature.asc
Description: PGP signature
st one symbol you had to set manually ...
The root of the issue is that we have no way to change dynamically the
set of flags used in pg_config.h.win32 with the MSVC scripts depending
on the version of OpenSSL compiled. We can tweak the scripts to do that
automatically Patches we
This may also be of interest to you, but checking official docs as Adrian
recommended is best.
https://why-upgrade.depesz.com/show?from=9.4.2&to=9.6.12&keywords=
I have not personally used this, but the write-up seems solid to minimize
downtime to help you to shift data gradually. Be sure you understand the
limitations of partitioning, particularly when you are still on 10x not yet
on v11 where updates will shift a row to a new partition if the partition
ke
ustom value per standby.
--
Michael
signature.asc
Description: PGP signature
A cron job will only run once a minute, not wake up every second. But you
could write a PHP program that does a one-second sleep before checking if
there's something to do, and a batch job that runs periodically to see if
the PHP program is running, and if not, launch it.
That's how I handle a jo
ion from 3 to 2.
--
Michael
signature.asc
Description: PGP signature
es:
https://www.postgresql.org/download/windows/
--
Michael
signature.asc
Description: PGP signature
Hello,
We have a database cluster which recently got very close to XID Wraparound. To
get
it back under control I've been running a lot of aggressive manual vacuums.
However, I have noticed a few anomolies. When I try to check the status of
vacuum commands:
qtodb_pmxtr=# select * from pg_stat
On Tue, Jul 23, 2019 at 1:36 PM Adrian Klaver
wrote:
> On 7/23/19 12:20 PM, PegoraroF10 wrote:
> > We have in a single database 190 identical schemas. Now, when we create
> a new
> > one, with exactly same structure as the previous ones, it takes 20 or 30
> > minutes to finish. Usual time to fini
As a note to the original poster, you might want to check out-
https://www.postgresql.org/docs/current/citext.html
uot;.
So doing a dump from a 9.5 instance using pg_dump from 9.6 would have
set the parameter.
--
Michael
signature.asc
Description: PGP signature
A few years ago I tried to take an app someone had written for us in
Access years ago and convert it to Postgres.
It seemed like for every rule I tried there were a handful of exceptions.
We wound up just rewriting the app and not trying to export the data from
the previous one.
I hope your pro
Are you autovacuuming and analyzing aggressively enough? Is there bloat or
other concern for these system tables? I expect it may be a concern based
on what I remember about your environment having thousands of tables and
perhaps autovacuum not keeping up (re: Too slow to create new schema and
thei
If those conditions that are throwing off the stats are expected to be
minimally impactful/filtering few rows, then you can use the one
tried-and-true optimizer hint (aside from materialized CTEs, stylized
indexes, etc) --- OFFSET 0 at the end of a sub-query.
SELECT * FROM ( [your existing query w
-- I'm thinking the OFFSET 0 create an optimization barrier that prevents
the planner from collapsing that sub-query into the top query, and enforces
ordering in the query?
That's my understanding. I think it is an optimizer hint by another name. I
used to put things in a CTE (which is always mate
Why can I use an alias for group by and order by and not for having? I am
just wondering if there is some barrier to it being implemented, or if it
just hasn't been.
select
table_schema || '.' || table_name AS schema_qualified_table_name,
count( column_name ) as column_count
from
information_schem
What is default_statistics_target set to? Or is there custom statistics
value set for this table/column? Perhaps the planner is making sub-optimal
choices because it only has a vague idea about the data and the histogram
is not very informative. Planning time will increase when statistics target
is
Might the issue be with pg_index or pg_attribute rather than pg_class? Is
the query still slow? Do you have the output of explain analyze for use on
https://explain.depesz.com/ or the output of EXPLAIN (ANALYZE, COSTS,
VERBOSE, BUFFERS, FORMAT JSON) for use on
http://tatiyants.com/pev/#/plans/new w
Can you manually execute vacuum analyze on all three involved tables and
again share the plan for the same query? If it is significantly improved,
it would seem like following the recommendation to tune autovacuum (and
analyze) to be more frequent would be prudent.
You haven't seemed to change fro
On Tue, Aug 27, 2019 at 9:45 PM Laurenz Albe
wrote:
> Holtgrewe, Manuel wrote:
> > Switching off fsync leads to a drastic time improvement but still
> > higher wall-clock time for four threads.
>
> Don't do that unless you are ready to start from scratch with a new
> "initdb" in the case of a cra
You need the wildcard character at front and back.
select * from fish_counts where stream_tribs ilike '*%*Nehalem*%*';
>
> CREATE STATISTICS MT_ReservationDepartureUtc (ndistinct) on "AccountCode",
> "DepartureUtc" from "Reservation"."Reservation";
> CREATE STATISTICS MT_ReservationArrivalUtc (ndistinct) on "AccountCode",
> "ArrivalUtc" from "Reservation"."Reservation";
> CREATE STATISTICS MT_ReservationNoShowFeeId
I think I would expect this behavior with how you have defined the
constraints as the function results rather than just being strict
comparisons to the timestamp field.
Instead of this-
Check constraints:
> "y2018_mis_ora_check" CHECK (date_part('year'::text, mis_ora) =
> 2018::double precisio
1 - 100 of 888 matches
Mail list logo