Re: Issues with EPOCH-s, TIMESTAMP(TZ)-s and leap seconds.

2025-01-27 Thread Thomas Munro
On Tue, Jan 28, 2025 at 10:02 AM Nem Tudom wrote: > Any help, advice, recommendations, URL-s, references &c. appreciated. As others have said, we're using the POSIX AKA Unix time scale, as almost all general purpose computer systems do. It's based on the UTC time scale (the one that has SI secon

Re: Running docker in postgres, SHM size of the docker container in postgres 16

2024-11-19 Thread Thomas Munro
On Wed, Nov 20, 2024 at 11:22 AM Koen De Groote wrote: > Why would that be? It's the exact same data. The install is about 50GB in > size. Is there something wrong with postgres 16, or did some settings > significantly change, that I need to know about? I went over all the > changelogs, nothing

Re: Naive question about multithreading/multicore

2024-10-12 Thread Thomas Munro
On Sun, Oct 13, 2024 at 6:31 AM Marc SCHAEFER wrote: > template1=> SELECT COUNT(*) FROM pg_class a, pg_class b, pg_class c; > > I see only one 100% CPU PostgreSQL process. If you set set min_parallel_table_scan_size = 0 then it uses parallelism, and completes much faster. The planner generally w

Re: Getting PSQL in Windows to support TAB/Autocomplete via modified readline...

2024-10-07 Thread Thomas Munro
Hi Kirk, Just as an FYI since you were working in this area, Tom has just knocked off one of the blockers for tab completion on Windows, namely that MSVC didn't like the overgrown if-then-else code in tab-complete.c[1]. That is now fixed in PostgreSQL's master branch[2][3][4] (v18 to be). [1] h

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-12 Thread Thomas Munro
On Thu, Sep 12, 2024 at 8:54 PM Pecsök Ján wrote: > In link you provided there is mention, that in PostgreSQL 16 data is not being > compressed for PostgreSQL 16 server. Does it mean, that PosgreSQL 16 use much > more space while computing queries? > If that is the case, it can be our problem, be

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
I don't understand what ENOSPC has to do with the file descriptor limits, but this person reported: # touch test touch: cannot touch ‘test’: No space left on device https://serverfault.com/questions/746032/rsync-and-scp-failing-with-no-space-left-on-xfs-device ... with plenty of free space, and

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
On Thu, Sep 12, 2024 at 12:39 AM Alvaro Herrera wrote: >> On 2024-Sep-11, Pecsök Ján wrote: > > In our case: > > Kernel: Linux version 4.18.0-513.18.1.el8_9.ppc64le > > (mockbu...@ppc-hv-13.build.eng.rdu2.redhat.com) (gcc version 8.5.0 20210514 > > (Red Hat 8.5.0-20) (GCC)) #1 SMP Thu Feb 1 02:5

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
On Wed, Sep 11, 2024 at 9:56 PM Alvaro Herrera wrote: > On 2024-Sep-10, Pecsök Ján wrote: > > After upgrade of Posgres from version 13.5 to 16.2 we experience following > > error: > > could not extend file "pg_tblspc/16401/PG_16_202307071/17820/3968302971" > > with FileFallocate(): No space left

Re: Windows installation problem at post-install step

2024-08-07 Thread Thomas Munro
Thanks. The log didn't offer any more clues, and my colleague David R has Windows and knows how to work its debugger so we sat down together and chased this down (thanks David!). 1. It is indeed calling abort(), but it's not a PANIC or Assert() in PostgreSQL, it's an assertion inside Windows' ow

Re: Windows installation problem at post-install step

2024-08-06 Thread Thomas Munro
On Tue, Aug 6, 2024 at 11:44 PM Peter J. Holzer wrote: > I assume that "1254" here is the code page. > But you specified --encoding=UTF-8 above, so your default locale uses a > different encoding than the template databases. I would expect that to > cause problems if the template databases contain

Re: Windows installation problem at post-install step

2024-08-06 Thread Thomas Munro
On Tue, Aug 6, 2024 at 10:38 PM Sandeep Thakkar wrote: > On Tue, Aug 6, 2024 at 4:06 PM Sandeep Thakkar > wrote: [v15] >>> XXX debug raw: setup_locale_encoding = "Turkish_Türkiye.1254" >>> XXX debug hex: setup_locale_encoding = { 54 75 72 6b 69 73 68 5f 54 fc 72 >>> 6b 69 79 65 2e 31 32 35

Re: Windows installation problem at post-install step

2024-08-05 Thread Thomas Munro
less confusing than looking at script output via email (I don't even know how many onion layers of transcoding are involved...) From b97fe5a55e50a447d41a439412922ffe3f7e168b Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Tue, 6 Aug 2024 16:06:29 +1200 Subject: [PATCH 1/3] xxx debug --- src/

Re: Windows installation problem at post-install step

2024-07-22 Thread Thomas Munro
On Mon, Jul 22, 2024 at 11:51 PM Sandeep Thakkar wrote: > EDB's windows installer gets the locales on the system using the > https://github.com/EnterpriseDB/edb-installers/blob/REL-16/server/scripts/windows/getlocales/getlocales.cpp > and then substitute some patterns > (https://github.com/Ente

Re: Windows installation problem at post-install step

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 11:58 AM Ertan Küçükoglu wrote: > Thomas Munro , 21 Tem 2024 Paz, 23:27 tarihinde şunu > yazdı: >> 2. Some existing database clusters which had been installed with the >> name "Turkish_Turkey.1254" became unstartable when the OS upgra

Re: Windows installation problem at post-install step

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 7:29 AM Adrian Klaver wrote: > On 7/21/24 12:00, Ertan Küçükoglu wrote: > > My main purpose was and still is to reach EDB people using the forum and > > let them know about the problem. > > I believe it is something to be fixed for future installations. I would > > like to

Re: libpq v17 PQsocketPoll timeout is not granular enough

2024-06-10 Thread Thomas Munro
On Tue, Jun 11, 2024 at 2:36 AM Dominique Devienne wrote: > Hi. I've noticed [that libpq API in v17 beta1][1], and wanted to use > it to replace an existing Boost.ASIO-based async polling of the > connection's socket, waiting for notifications. The use case being > using PostgreSQL LISTEN/NOTIFY f

Re: Preallocation changes in Postgresql 16

2024-04-25 Thread Thomas Munro
On Fri, Apr 26, 2024 at 4:37 AM Riku Iki wrote: > I am wondering if there were preallocation related changes in PG16, and if it > is possible to disable preallocation in PostgreSQL 16? I have no opinion on the btrfs details, but I was wondering if someone might show up with a system that doesn't

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-04-03 Thread Thomas Munro
On Sat, Mar 23, 2024 at 3:01 AM Nick Renders wrote: > We now have a second machine with this issue: it is an Intel Mac mini running > macOS Sonoma (14.4) and PostgreSQL 16.2. > This one only has a single Data directory, so there are no multiple instances > running. BTW if you're running databas

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-04-03 Thread Thomas Munro
On Thu, Apr 4, 2024 at 3:11 AM Nick Renders wrote: > In the macOS Finder, when you show the Info (command+i) for an external drive > (or any partition that is not the boot drive), there is a checkbox "Ignore > ownership on this volume" in the Permissions section. I think it is by > default "on"

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-03-28 Thread Thomas Munro
On Fri, Mar 29, 2024 at 4:47 AM Nick Renders wrote: > Looking at the 2 machines that are having this issue (and the others that > don't), I think it is somehow related to the following setup: > - macOS Sonoma (14.4 and 14.4.1) > - data directory on an external drive > > That external drive (a Pro

Re: How do you optimize the disk IO when you cannot assume a file will start at a boundary then?

2024-02-22 Thread Thomas Munro
working on bringing to PostgreSQL in useful form, we had to do this: commit faeedbcefd40bfdf314e048c425b6d9208896d90 Author: Thomas Munro Date: Sat Apr 8 10:38:09 2023 +1200 Introduce PG_IO_ALIGN_SIZE and align all I/O buffers. ... to avoid EINVAL errors, falling back to buffered mode or pa

Re: Query crash with 15.5 on debian bookworm/armv8

2023-12-26 Thread Thomas Munro
On Wed, Dec 27, 2023 at 5:17 AM Clemens Eisserer wrote: > > FWIW, since this crash is inside LLVM you could presumably dodge the bug > > by setting "jit" to off. > > Thanks, this indeed solved the crash. > Just to make sure this crash doesn't have anything to do with my > setup/config (I'd changed

Re: How to generate random bigint

2023-12-20 Thread Thomas Munro
On Thu, Dec 21, 2023 at 7:21 PM Tom Lane wrote: > Phillip Diffley writes: > > Postgres's random() function generates a random double. That can be > > converted to a random int for smaller integers, but a double can't > > represent all of the values in a bigint. Is there a recommended way to > > g

Re: psql crash with custom build on RedHat 7

2023-12-19 Thread Thomas Munro
On Wed, Dec 20, 2023 at 4:41 AM Dominique Devienne wrote: > On Tue, Dec 19, 2023 at 2:02 PM Thomas Munro wrote: >> On Wed, Dec 20, 2023 at 1:39 AM Dominique Devienne >> wrote: >> > Program received signal SIGSEGV, Segmentation fault. >> > 0x004232b8 in

Re: psql crash with custom build on RedHat 7

2023-12-19 Thread Thomas Munro
On Wed, Dec 20, 2023 at 1:39 AM Dominique Devienne wrote: > Program received signal SIGSEGV, Segmentation fault. > 0x004232b8 in slash_yylex () I think this might have to do with flex changing. Does it help if you "make maintainer-clean"?

Re: fsync data directory after DB crash

2023-07-20 Thread Thomas Munro
On Wed, Jul 19, 2023 at 2:09 PM Pandora wrote: > Yes, I saw the usage of syncfs in PG14, but it is recommended to use it on > Linux 5.8 or higher. If my OS version is lower than 5.8, can I still enable > it? Nothing stops you from enabling it, it's fairly ancient and should work. It just doesn

Re: fsync data directory after DB crash

2023-07-18 Thread Thomas Munro
On Wed, Jul 19, 2023 at 12:41 PM Michael Paquier wrote: > On Tue, Jul 18, 2023 at 04:50:25PM +0800, Pandora wrote: > > I found that starting from version 9.5, PostgreSQL will do fsync on > > the entire data directory after DB crash. Here's a question: if I > > have FPW = on, why is this step still

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-08 Thread Thomas Munro
On Tue, May 9, 2023 at 10:04 AM Tom Lane wrote: > Michael Paquier writes: > > One thing I was wondering about to improve the odds of the hits is to > > be more aggressive with the number of relations created at once, so as > > we are much more aggressive with the number of pages extended in > > p

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Mon, May 8, 2023 at 2:24 PM Michael Paquier wrote: > I can reproduce the same backtrace here. That's just my usual laptop > with ext4, so this would be a Postgres bug. First, here are the four > things running in parallel so as I can get a failure in loading a > critical index when connecting

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Mon, May 8, 2023 at 4:10 AM Evgeny Morozov wrote: > On 6/05/2023 11:13 pm, Thomas Munro wrote: > > Would you like to try requesting FILE_COPY for a while and see if it > > eventually happens like that too? > Sure, we can try that. Maybe you could do some one way and some

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Sun, May 7, 2023 at 1:21 PM Tom Lane wrote: > Thomas Munro writes: > > Did you previously run this same workload on versions < 15 and never > > see any problem? 15 gained a new feature CREATE DATABASE ... > > STRATEGY=WAL_LOG, which is also the default. I wond

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sun, May 7, 2023 at 10:23 AM Jeffrey Walton wrote: > This may be related... I seem to recall the GNUlib folks talking about > a cp bug on sparse files. It looks like it may be fixed in coreutils > release 9.2 (2023-03-20): > https://github.com/coreutils/coreutils/blob/master/NEWS#L233 > > If I

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sun, May 7, 2023 at 12:29 AM Evgeny Morozov wrote: > On 6/05/2023 12:34 pm, Thomas Munro wrote: > > So it does indeed look like something unknown has replaced 32KB of > > data with 32KB of zeroes underneath us. Are there more non-empty > > files that are all-zeroes? Some

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sat, May 6, 2023 at 9:58 PM Evgeny Morozov wrote: > Right - I should have realised that! base/1414389/2662 is indeed all > nulls, 32KB of them. I included the file anyway in > https://objective.realityexists.net/temp/pgstuff2.zip OK so it's not just page 0, you have 32KB or 4 pages of all zero

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-05 Thread Thomas Munro
On Fri, May 5, 2023 at 7:50 PM Evgeny Morozov wrote: > The OID of the bad DB ('test_behavior_638186279733138190') is 1414389 and > I've uploaded base/1414389/pg_filenode.map and also base/5/2662 (in case > that's helpful) as https://objective.realityexists.net/temp/pgstuff1.zip Thanks. That pg

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 11:15 AM Thomas Munro wrote: > What does select > pg_relation_filepath('pg_class_oid_index') show in the corrupted > database, base/5/2662 or something else? Oh, you can't get that far, but perhaps you could share the pg_filenode.map file? Or

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 11:15 AM Thomas Munro wrote: > Now *that* is a piece of > logic that changed in PostgreSQL 15. It changed from sector-based > atomicity assumptions to a directory entry swizzling trick, in commit > d8cd0c6c95c0120168df93aae095df4e0682a08a. Hmm. I spoke to

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 6:11 AM Evgeny Morozov wrote: > Meanwhile, what do I do with the existing server, though? Just try to > drop the problematic DBs again manually? That earlier link to a FreeBSD thread is surely about bleeding edge new ZFS stuff that was briefly broken then fixed, being disco

Re: src/test/examples/testlibpq2.c where the HAVE_SYS_SELECT_H is defined.

2023-03-17 Thread Thomas Munro
On Fri, Mar 17, 2023 at 7:48 PM jian he wrote: > Hi, > playing around with $[0] testlibpq2.c example. I wondered where > HAVE_SYS_SELECT_H is defined? > > I searched on the internet, founded that people also asked the same question > in $[1]. > > In my machine, I do have . > system version: Ubun

Re: 13.x, stream replication and locale(?) issues

2023-03-03 Thread Thomas Munro
On Wed, Mar 1, 2023 at 10:30 AM Thomas Munro wrote: > On Wed, Mar 1, 2023 at 12:09 AM Eugene M. Zheganin wrote: > > 3) how do I fix it ? Should I take locale sources for ru_RU.utf8 on Linux > > and compile it on FreeBSD - will it help ? > > Out of curiosity (I'm n

Re: Interval in hours but not in days Leap second not taken into account

2023-02-28 Thread Thomas Munro
On Mon, Feb 27, 2023 at 8:26 PM PALAYRET Jacques wrote: > # PostgreSQL does not take into account the additional second (leap second) > in some calendar days ; eg. 2016, 31 dec. : > SELECT to_timestamp('20170102 10:11:12','mmdd hh24:mi:ss') - > to_timestamp('20161230 00:00:00','mmdd hh24

Re: 13.x, stream replication and locale(?) issues

2023-02-28 Thread Thomas Munro
On Wed, Mar 1, 2023 at 12:09 AM Eugene M. Zheganin wrote: > 3) how do I fix it ? Should I take locale sources for ru_RU.utf8 on Linux and > compile it on FreeBSD - will it help ? Out of curiosity (I'm not saying it's a good idea!), do you know if FreeBSD's localedef can compile glibc's collatio

Re: Getting PSQL in Windows to support TAB/Autocomplete via modified readline...

2022-11-22 Thread Thomas Munro
On Tue, Nov 22, 2022 at 4:25 AM Kirk Wolak wrote: > In researching this problem, it appears that the decision was made like > 17yrs ago, when windows did not have a realistic "terminal" type interface. > Assuming we target Windows 8.1 or higher, I believe this goes away. FWIW PostgreSQL 16 w

Re: PANIC: could not flush dirty data: Cannot allocate memory

2022-11-15 Thread Thomas Munro
On Wed, Nov 16, 2022 at 1:24 AM wrote: > Filesystem is ext4. VM technology is mixed: VMware, KVM and XEN PV. > Kernel is 5.15.0-52-generic. > > We have not seen this with Ubutnu 18.04 and 20.04 (although we might not > have noticed it). > > I guess upgrading to postgresql 13/14/15 does not help as

Re: PANIC: could not flush dirty data: Cannot allocate memory

2022-11-14 Thread Thomas Munro
On Tue, Nov 15, 2022 at 10:54 AM Christoph Moench-Tegeder wrote: > ## klaus.mailingli...@pernau.at (klaus.mailingli...@pernau.at): > > On several servers we see the error message: PANIC: could not flush > > dirty data: Cannot allocate memory > Of these three places, there's an sync_file_range(),

Re: Segmentation Fault PG 14

2022-11-07 Thread Thomas Munro
On Tue, Nov 8, 2022 at 11:45 AM Willian Colognesi wrote: > root@ip-10-x-x-x:/home/ubuntu# pg_config --configure > ... --with-extra-version= (Ubuntu 14.5-2.pgdg20.04+2)' ... > ... '--with-llvm' 'LLVM_CONFIG=/usr/bin/llvm-config-10' ... > There is no llvm installed on ubuntu server, postgresql was

Re: Strange collation names ("hu_HU.UTF-8")

2022-08-02 Thread Thomas Munro
On Wed, Aug 3, 2022 at 1:43 AM Tom Lane wrote: > I believe most if not all variants of Unix are > permissive about the spelling of the encoding part. I've only seen glibc doing that downcase-and-strip-hyphens thing to the codeset part of a locale name when looking for locale definition files. Ot

Re: AIX and EAGAIN on open()

2022-07-03 Thread Thomas Munro
On Mon, Jun 20, 2022 at 9:53 PM Christoph Berg wrote: > IBM's reply to the issue back in December 2020 was this: > > The man page / infocenter document is not intended as an exhaustive > list of all possible error codes returned and their circumstances. > "Resource temporarily unavailable" m

Re: Order of rows in statement triggers NEW/OLD tables

2022-05-31 Thread Thomas Munro
On Fri, May 6, 2022 at 6:20 PM hubert depesz lubaczewski wrote: > when defining statement triggers on update I can use: > > REFERENCING OLD TABLE AS xxx NEW TABLE as YYY > > these "pseudo" tables contain rows that were before and after. > > Is the order guaranteed? > > Can I assume that "first" ro

Re: Improve configurability for IO related behavoir

2022-05-28 Thread Thomas Munro
On Sun, May 29, 2022 at 4:29 AM 浩辰 何 wrote: > Furthermore, the results above are also related to IO API supported by OS. > MySQL support synchronized IO and Linux libaio. It seems > that PostgreSQL only supports synchronized IO, so shall we support more IO > engines? like io_uring which is very

Re: Pg14 possible index corruption after reindex concurrently

2022-05-24 Thread Thomas Munro
On Wed, May 25, 2022 at 6:17 AM Aleš Zelený wrote: > SELECT format('REINDEX SCHEMA CONCURRENTLY %I;', n.nspname) This may be related to bug #17485, discussed at: https://www.postgresql.org/message-id/flat/17485-396609c6925b982d%40postgresql.org

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Thomas Munro
On Sun, Apr 24, 2022 at 8:00 AM Peter wrote: > More than a million files in a single directory, this is > inacceptable. You didn't show EXPLAIN (ANALYZE) but if [Parallel] Hash is making insane numbers of temporary files then something is not working as intended... and I can take a guess at what

Re: frequency of positive and negative numbers, sizes of numbers and frequency of alteration of polarity

2022-02-17 Thread Thomas Munro
On Fri, Feb 18, 2022 at 10:42 AM Shaozhong SHI wrote: > Given 2 or more such columns, is there any measure that can be calculated to > tell which one alternates more than others? Well, you could report non-flips as NULL and flips as magnitude, and then wrap that query in another query to compute

Re: frequency of positive and negative numbers, sizes of numbers and frequency of alteration of polarity

2022-02-17 Thread Thomas Munro
On Fri, Feb 18, 2022 at 9:11 AM Shaozhong SHI wrote: > How to calculate frequency of positive and negative numbers and define and > calculate frequency of alteration of polarity? > > Surely, we can use frequency of alteration of polarity and level of change > (e.g., size of positive and negative

Re: Compile 14.1 in EL5.8

2022-02-09 Thread Thomas Munro
On Thu, Feb 10, 2022 at 2:23 PM Tom Lane wrote: > Thomas Munro writes: > > ... I wondered about also removing the leftover comment > > "We assume that any system that has Linux epoll() also has Linux > > signalfd()" which was my attempt to explain that there wasn&#

Re: Compile 14.1 in EL5.8

2022-02-09 Thread Thomas Munro
On Thu, Feb 10, 2022 at 3:38 AM Tom Lane wrote: > Gabriela Serventi writes: > > Hi! I'm trying to compile release 14.1 in a very old Linux Server (Red Hat > > Enterprise Linux Server release 5.8). I can run configure successfully, but > > when I try to run make, I get the following error: > > lat

Re: sort order for UTF-8 char column with Japanese UTF-8

2022-02-03 Thread Thomas Munro
On Fri, Feb 4, 2022 at 8:11 AM Matthias Apitz wrote: > On my FreeBSD laptop the same file sorts as > > guru@c720-r368166:~ $ LANG=de_DE.UTF-8 sort swd > A > ゲアハルト・A・リッター > ゲルハルト・A・リッター > チャールズ・A・ビアード > A010STRUKTUR > A010STRUKTUR > A010STRUKTUR > A0150SUPRALEITER Wow, so it's one thing to have a

Re: could not accept SSL connection: Success

2022-01-19 Thread Thomas Munro
On Thu, Jan 20, 2022 at 12:06 AM Carla Iriberri wrote: > On Wed, Jan 19, 2022 at 5:42 AM Michael Paquier wrote: >> "On an unexpected EOF, versions before OpenSSL 3.0 returned >> SSL_ERROR_SYSCALL, nothing was added to the error stack, and errno was >> 0. Since OpenSSL 3.0 the returned error is SS

Re: create database hangs forever on WSL - autovacuum deadlock?

2022-01-09 Thread Thomas Munro
On Sun, Jan 9, 2022 at 2:15 PM Alicja Kucharczyk wrote: > sob., 8 sty 2022 o 22:40 Thomas Munro napisał(a): >> On Sat, Jan 8, 2022 at 8:11 PM Alicja Kucharczyk >> wrote: >> > this is postgres running on windows wsl: PostgreSQL 14.1 (Ubuntu >> > 14.1-1.pg

Re: create database hangs forever on WSL - autovacuum deadlock?

2022-01-08 Thread Thomas Munro
On Sat, Jan 8, 2022 at 8:11 PM Alicja Kucharczyk wrote: > this is postgres running on windows wsl: PostgreSQL 14.1 (Ubuntu > 14.1-1.pgdg18.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu > 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit Is this WSL1 (some kind of Linux system call emulator running o

Re: Need to know more about pg_test_fsync utility

2021-12-12 Thread Thomas Munro
On Mon, Dec 13, 2021 at 3:04 PM PGSQL DBA wrote: > As you mentioned in question-8, "I'd investigate whether data is being cached > unexpectedly, perhaps indicating that committed transactions be lost in a > system crash event." So, I would like to know that if we configure the disk > for the WA

Re: Need to know more about pg_test_fsync utility

2021-12-09 Thread Thomas Munro
On Fri, Dec 10, 2021 at 3:20 PM PGSQL DBA wrote: > 1) How to interpret the output of pg_test_fsync? The main interesting area is probably the top section that compares the different wal_sync_method settings. For example, it's useful to verify the claim that fdatasync() is faster than fsync() (be

Re: Wrong sorting on docker image

2021-10-16 Thread Thomas Munro
On Sun, Oct 17, 2021 at 4:42 AM Tom Lane wrote: > Speaking of ICU, if you are using an ICU-enabled Postgres build, > maybe you could find an ICU collation that acts the way you want. > This wouldn't be a perfect solution, because we don't yet have > the ability to set an ICU collation as a databas

Re: To JIT (or not!) in Ubuntu packages

2021-09-14 Thread Thomas Munro
On Wed, Sep 15, 2021 at 3:30 PM Ben Chobot wrote: > So I've installed > http://apt.postgresql.org/pub/repos/apt/pool/14/p/postgresql-14/postgresql-14_14~beta3-1.pgdg18.04+1_arm64.deb, > after which I see: Ahhh, so you're on 18.04, an old LTS. I remember now, there was this issue in LLVM 3.9 on

Re: To JIT (or not!) in Ubuntu packages

2021-09-14 Thread Thomas Munro
On Tue, Sep 14, 2021 at 10:11 AM Ben Chobot wrote: > We've noticed that the Ubuntu postgresql-12 package has --with-llvm > enabled on x86_64, but not on aarch64. Does anybody know if this was > intentional, or just an oversight? > > For what it's worth, it seems the beta postgresql-14 package for

Re: spiexceptions.UndefinedFile: could not open file "base/16482/681830": No such file or directory

2021-09-08 Thread Thomas Munro
On Thu, Sep 9, 2021 at 9:19 AM Celia McInnis wrote: > Note that the file does exist:! (How do I know if it is looking under the > correct directory? Other times I have done similar temporary table creations > with no problems!): PostgreSQL internally uses relative paths. It's probably not a ve

Re: Is there something similar like flashback query from Oracle planned for PostgreSQL

2021-06-23 Thread Thomas Munro
On Thu, Jun 24, 2021 at 6:54 AM Dirk Krautschick wrote: > Is there something planned to get a behaviour like Oracle's flashback query > based on the old values > before deleted by vacuum? > > So a feature to recreate old versions of rows if still there? > > Or are there any related extensions or

Re: order by

2021-06-10 Thread Thomas Munro
On Thu, Jun 10, 2021 at 8:40 PM Luca Ferrari wrote: > On Thu, Jun 10, 2021 at 10:15 AM Vijaykumar Jain > > when you provide a table in query in the order by clause, it is > > ordered by cols of that table in that order. > > Clever, thanks! > I also realized that this "table to tuples" expansion wo

Re: CREATE/REFRESH MATERIALIZED VIEW planner difference?

2021-06-01 Thread Thomas Munro
On Wed, Jun 2, 2021 at 7:15 AM Vijaykumar Jain wrote: > i only get workers to create mv, but refresh mv plan does not use workers for > the same conf params. Yeah, this changed in v14: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9e7ccd9ef64d05e87ceb1985d459bef9031205c0

Re: Chain Hashing

2021-05-06 Thread Thomas Munro
On Thu, May 6, 2021 at 9:48 PM Jian He wrote: > The following part is about the Chain Hashing. >> >> Maintain a linked list of buckets for each slot in the hash table. >> Resolve collisions by placing all elements with the same hash key into the >> same bucket. >> → To determine whether an elem

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Wed, May 5, 2021 at 2:12 PM Thomas Munro wrote: > It might be interesting to know how that 40ms time scales as you add > more workers. ... Another thought: I'd also try tests like that in large databases (ie large virtual memory) vs small ones, and with and without huge/locked m

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Wed, May 5, 2021 at 3:50 AM Hans Buschmann wrote: > (BTW: Is this cost multiplied by the real count of workers choosen > (max_parallel_workers_per_gather) or only a value independent of the number > of workers?. This would matter in windows-high-parallel scenarios) It's not multiplied: http

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Tue, May 4, 2021 at 7:40 PM Hans Buschmann wrote: > The problem seems that this (probably inherent) performance disadvantage of > windows is not reflected in the cost model. https://www.postgresql.org/docs/13/runtime-config-query.html#GUC-PARALLEL-SETUP-COST is for that. It might be interest

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-03 Thread Thomas Munro
On Tue, May 4, 2021 at 4:05 AM Hans Buschmann wrote: > The main difference is the time shown for the Gather Merge step (65 ms vs. 7 > ms) No Windows here, but could it be super slow at launching workers? How does a trivial parallel query compare, something like? SET force_parallel_mode = on; E

Re: -1/0 virtualtransaction

2021-04-30 Thread Thomas Munro
On Wed, Apr 28, 2021 at 2:25 AM Mike Beachy wrote: > On Tue, Apr 27, 2021 at 2:56 AM Laurenz Albe wrote: >> >> Not sure, but do you see prepared transactions in "pg_prepared_xacts"? > > No, the -1 in the virtualtransaction > (https://www.postgresql.org/docs/11/view-pg-locks.html) for pg_prepared

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Thomas Munro
On Mon, Mar 15, 2021 at 1:29 PM Avinash Kumar wrote: > Is this expected when replication is happening between PostgreSQL databases > hosted on different OS versions like Ubuntu 16 and Ubuntu 20 ? Or, do we > think this is some sort of corruption ? Is this index on a text datatype, and using a c

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-11 Thread Thomas Munro
Hi Andrus, On Thu, Mar 11, 2021 at 2:21 AM Andrus wrote: > Windows Resource manger shows that wal files are used by large number of > postgres processes: > > postgres.exe22656FileC:\Program > Files\PostgreSQL\13\data\pg_wal\0001000A0075 > postgres.exe30788File

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-09 Thread Thomas Munro
On Tue, Mar 9, 2021 at 9:43 PM Andrus wrote: > > Any hints in Windows event viewer? Events occurring at the same time > showing up there. > > Looked into Administrative Events/Custom views and few others. There are no > messages about this. Windowsi perfomance monitor and Filemon show files >

Re: Log files polluted with permission denied error messages after every 10 seconds

2021-03-08 Thread Thomas Munro
On Sat, Mar 6, 2021 at 2:36 PM Michael Paquier wrote: > On Fri, Mar 05, 2021 at 07:36:37PM +0200, Andrus wrote: > > Then turned real-time protection off: > > > > Problem persists. New entry is written after every 10 seconds. > > On which files are those complaints? It seems to me that you may hav

Re: [LDAPS] Test connection user with ldaps server

2021-02-15 Thread Thomas Munro
On Tue, Feb 16, 2021 at 4:32 AM Laurenz Albe wrote: > What I would do is experiment with the "ldapsearch" executable from OpenLDAP > and see > if you can reproduce the problem from the command line. Also, maybe try doing this as the "postgres" user (or whatever user PostgreSQL runs as), just in

Re: How to post to this mailing list from a web based interface

2021-02-14 Thread Thomas Munro
On Fri, Jan 29, 2021 at 4:27 AM Alvaro Herrera wrote: > On 2021-Jan-28, Ravi Krishna wrote: > > I recollect there use to be a website from where one can reply from web. > > The community does not maintain such a service. > > There used to be a Gmane archive of this list that you could use to > pos

Re: Unable To Drop Tablespace

2021-02-04 Thread Thomas Munro
On Fri, Feb 5, 2021 at 12:43 PM Ian Lawrence Barwick wrote: > 2021年2月5日(金) 3:52 Pavan Pusuluri : >> We are trying to drop a table space on RDS Postgres . We have removed the >> objects etc, but it still won't drop. >> >> I have checked and there's no reference anywhere to this tablespace but it

Re: Unable to compile postgres 13.1 on Slackware current x64

2020-11-17 Thread Thomas Munro
On Tue, Nov 17, 2020 at 8:02 PM Condor wrote: > I try to compile postgres again with (cd src/backend/commands; sed > 's/TRUE/true/' collationcmds.c > collationcmds.c; ) and it's compiled > but get new error on linking: Doesn't that produce an empty file collationcmds.c? I think you want: sed 's/

Re: Unable to compile postgres 13.1 on Slackware current x64

2020-11-16 Thread Thomas Munro
On Mon, Nov 16, 2020 at 10:10 PM Laurenz Albe wrote: > On Mon, 2020-11-16 at 09:15 +0200, Condor wrote: > > collationcmds.c: In function ‘get_icu_language_tag’: > > collationcmds.c:467:51: error: ‘TRUE’ undeclared (first use in this > > function); did you mean ‘IS_TRUE’? > >467 | uloc_toLangu

Re: PostgreSQL on Windows' state

2020-09-23 Thread Thomas Munro
On Wed, Sep 23, 2020 at 10:53 PM Alessandro Dentella wrote: > Thanks Pavel, but psycopg2 (that I always use is just for Python). T > hey claim/complain that from c# there's no native solution. Maybe https://www.npgsql.org/?

Re: Check replication lag

2020-08-05 Thread Thomas Munro
On Thu, Aug 6, 2020 at 7:02 AM Sreejith P wrote: > IN SYNC mode of replication what would be the impact on Master DB server in > terms of over all performance ? The pg_stat_replication columns write_lag, flush_lag and replay_lag are designed tell you how long to expect commits to take for synch

Re: PG 9.5.5 cores on AIX 7.1

2020-07-19 Thread Thomas Munro
On Sun, Jul 19, 2020 at 11:01 PM Abraham, Danny wrote: > Segmentation fault in _alloc_initial_pthread at 0x9521474 > 0x9521474 (_alloc_initial_pthread+0x1d4) e803 ld > r0,0x0(r3) > (dbx) where > _alloc_initial_pthread(??) at 0x9521474 > __pth_init(??) at

Re: Same query taking less time in low configuration machine

2020-07-15 Thread Thomas Munro
On Tue, Jul 14, 2020 at 9:27 PM Vishwa Kalyankar wrote: > Function Scan on kseb_geometry_trace_with_barrier_partition > (cost=0.25..10.25 rows=1000 width=169) (actual time=11626.548..11626.568 > rows=254 loops=1) > Function Scan on kseb_geometry_trace_with_barrier_partition > (cost=0.25..1

Re: Definition of REPEATABLE READ

2020-06-21 Thread Thomas Munro
On Mon, Jun 22, 2020 at 12:25 AM Peter J. Holzer wrote: > I've read http://jepsen.io/analyses/postgresql-12.3 which reports a > problem in PostgreSQL's SERIALIZABLE isolation leven (which has since been > fixed) and also shows an example of a violation of what they consider to > be the correct def

Re: troubleshooting postgresql ldap authentication

2020-06-08 Thread Thomas Munro
On Tue, Jun 9, 2020 at 9:05 AM Chris Stephens wrote: > hostsslall all 0.0.0.0/0 ldap > ldapserver="ldaps://xxx" ldapbasedn="yyy" ldaptls=1 > does anyone know what might be causing "LDAP: Bad parameter to an ldap > routine" You probably want ldapurl="lda

Re: Shared memory error

2020-06-04 Thread Thomas Munro
On Fri, Jun 5, 2020 at 1:00 AM Sonam Sharma wrote: > The dynamic_shared_memory_type was set to POSIX . Because of this it was > using tmpfs /dev/shm. When the query was running I saw the file system was > filling. So I extended the file system and luckily the query worked for that > time Oh,

Re: Shared memory error

2020-06-04 Thread Thomas Munro
>> Do you see any other errors around this one, in the PostgreSQL logs? > No , only this is the error from db and jdbc end .. and queries are failing If you need a workaround right now you could always set max_parallel_workers_per_gather=0 so that it doesn't try to use parallel query. That could

Re: Shared memory error

2020-06-04 Thread Thomas Munro
On Thu, Jun 4, 2020 at 6:18 AM Sonam Sharma wrote: >>> 1) Postgres version : 11.2 FYI This is missing over a year's worth of bugfixes. That said, I don't know of anything fixed that has this symptom. >>> 4) Is this only with one query and if so what is it doing? : No , few >>> queries work, fe

Re: Can we get SQL Server-like cross database queries

2020-06-03 Thread Thomas Munro
On Thu, Jun 4, 2020 at 4:26 PM Ron wrote: > On 6/3/20 2:57 PM, Rob Sargent wrote: > >> On Jun 3, 2020, at 1:46 PM, Tom Lane wrote: > >> Guyren Howe writes: > >>> Is it practical to provide the SQL Server-like feature in Postgres? > >> No. > > That got me chuckling. > > I had just decided not to

Re: Help understanding SIReadLock growing without bound on completed transaction

2020-05-22 Thread Thomas Munro
On Fri, May 22, 2020 at 7:48 AM Mike Klaas wrote: > It's my understanding that these locks should be cleared when there are no > conflicting transactions. These locks had existed for > 1 week and we have > no transactions that last more than a few seconds (the oldest transaction in > pg_stat_a

Re: Help understanding SIReadLock growing without bound on completed transaction

2020-05-21 Thread Thomas Munro
On Fri, May 22, 2020 at 7:48 AM Mike Klaas wrote: > locktype: page > relation::regclass::text: _pkey > virtualtransaction: 36/296299968 > granted:t > pid:2263461 That's an unusually high looking pid. Is that expected, for example did you crank Linux's pid_max right up, or is this AIX, or somethi

Re: Ubuntu 20.04: apt.postgresql.org/pub/repos/apt focal-pgdg InRelease' doesn't support architecture 'i386'

2020-05-16 Thread Thomas Munro
On Sun, May 17, 2020 at 10:45 AM Hugh wrote: > While this doesn't appear to be a bug that causes problems of any kind, I do > have a question about its cause. > > The "error" listed in the Subject: line is basically what I'm seeing. The > entire message is below, particularly the 'N:' at the end

Re: Is there a significant difference in Memory settings between 9.5 and 12

2020-05-11 Thread Thomas Munro
On Tue, May 12, 2020 at 2:52 PM Tory M Blue wrote: > It took the change but didn't help. So 10GB of shared_buffers in 12 is still > a no go. I'm down to 5GB and it works, but this is the same hardware, the > same exact 9.5 configuration. So I'm missing something. WE have not had to > mess with

Re: 12.2: Howto check memory-leak in worker?

2020-05-04 Thread Thomas Munro
On Tue, May 5, 2020 at 10:13 AM Peter wrote: > BTW, I would greatly appreciate if we would reconsider the need for > the server to read the postmaster.pid file every few seconds (probably > needed for something, I don't know). > That makes it necessary to set atime=off to get a spindown, and I > u

Re: Transition tables for column-specific UPDATE triggers

2020-05-03 Thread Thomas Munro
On Wed, Oct 9, 2019 at 3:59 PM Guy Burgess wrote: > The manual says: > https://www.postgresql.org/docs/current/sql-createtrigger.html > > A column list cannot be specified when requesting transition relations. > > And (I think the same point): > > The standard allows transition tables to be used

  1   2   3   >