[GENERAL] Spam on main page

2015-03-04 Thread pinker
You have spam on postgresql.org main page... "Jim Smith: Myśli o istotnych Szczegóły kwiatów dostawy online" It's not even proper polish :) -- View this message in context: http://postgresql.nabble.com/Spam-on-main-page-tp5840406.html Sent from the PostgreSQL - general mailing list archive at N

[GENERAL] VACUUM FULL doesn't reduce table size

2015-03-06 Thread pinker
I have deleted a large number of records from my_table, which originally had 288 MB. Then I ran vacuum full to make the table size smaller. After this operation size of the table remains the same, despite of the fact that table contains now only 241 rows and after rewriting it in classic way: CREAT

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-06 Thread pinker
Query output is empty... -- View this message in context: http://postgresql.nabble.com/VACUUM-FULL-doesn-t-reduce-table-size-tp5840782p5840797.html Sent from the PostgreSQL - general mailing list archive at Nabble.com. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-09 Thread pinker
Yes I have. Environment I'm working on is production snapshot, so there is no active transactions that could block those blocks from being removed... -- View this message in context: http://postgresql.nabble.com/VACUUM-FULL-doesn-t-reduce-table-size-tp5840782p5841085.html Sent from the Postgr

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-09 Thread pinker
Vick Khera wrote > On Fri, Mar 6, 2015 at 5:59 AM, pinker < > pinker@ > > wrote: > >> I have deleted a large number of records from my_table, which originally >> had >> 288 MB. Then I ran vacuum full to make the table size smaller. After this >> op

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-09 Thread pinker
I did: select pg_cancel_backend(pid) from pg_stat_activity where usename <> 'mine'; and then tried again vacuum full: INFO: vacuuming "my_table" INFO: "my_table": found 0 removable, 3043947 nonremovable row versions in 37580 pages DETAIL: 0 dead row versions cannot be removed yet. CPU 1.07s/1.

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-09 Thread pinker
Adrian Klaver-4 wrote > On 03/09/2015 07:08 AM, pinker wrote: >> I did: select pg_cancel_backend(pid) from pg_stat_activity where usename >> <> >> 'mine'; > > What makes you think that queries from usename = 'mine' are not important? > &

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-09 Thread pinker
select * from pg_stat_activity where state ilike 'idle in transaction%' it's empty. >And there is no relation between this table and the tables or functions being queried? no... >If snapshot is what I think it means, you might want to point them at: >http://www.postgresql.org/docs/9.3/intera

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-09 Thread pinker
and select txid_current_snapshot() - 1694632069:1694632069: select distinct xmin::text::bigint from sms_actual_mapper order by 1 desc 1694595273 -- View this message in context: http://postgresql.nabble.com/VACUUM-FULL-doesn-t-reduce-table-size-tp5840782p5841128.html Sent from the PostgreSQL

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-09 Thread pinker
> So there are no longer any dead rows being left behind, right? > > Why are we still discussing this? Do you have some other question? There are no dead rows, but postgres still cannot reuse the space because of 3043947 nonremovable row versions .. INFO: vacuuming "my_table" INFO: "my_table

Re: [GENERAL] VACUUM FULL doesn't reduce table size

2015-03-11 Thread pinker
Adrian Klaver-4 wrote > Also per Kevin Grittner and Tom Lane there > is a Nabble issue at work where the list here is not seeing all the > information. Example: It's because I edited some post. Then is visible only on nabble because edited version isn't sent back to the mailing list. Adrian Kl

[GENERAL] Error prone compilation of stored procedure

2015-07-06 Thread pinker
What's the reason behind very "tolerant" error checking during stored procedure compilation? Why PostgreSQL allows using variable (j_var) that doesn't exists? It isn't column name or isn't declared anywhere. Like in example below: CREATE OR REPLACE FUNCTION test() RETURNS int AS $BODY$ BEGIN s

Re: [GENERAL] Error prone compilation of stored procedure

2015-07-06 Thread pinker
Andreas Kretschmer-2 wrote > pinker < > pinker@ > > wrote:> What's the reason behind very "tolerant" error checking > during stored> procedure compilation?they are not compiled but interpreted > at runtime.Andreas Documentation says:

Re: [GENERAL] Error prone compilation of stored procedure

2015-07-06 Thread pinker
Pavel Stehule wrote > PLpgSQL doesn't check a identifiers inside embedded SQL before execution. > In this case j_var can be theoretically some SQL identifiers - the > possibility or impossibility is not know in function validation stage. > > It has some plus and minus points. The plus - there are

Re: [GENERAL] Error prone compilation of stored procedure

2015-07-06 Thread pinker
lup wrote > And are your colleagues offering to pay for an Oracle license? I would prefer to prove them it's not necessary :) -- View this message in context: http://postgresql.nabble.com/Error-prone-compilation-of-stored-procedure-tp5856699p5856734.html Sent from the PostgreSQL - general mail

[GENERAL] pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists

2015-07-07 Thread pinker
I made mistake in a filename in pg_dump command, i.e. have used path from another server, which not exists on this one. pg_dump instead of checking permissions / existence of output file first dumped the whole database and at the end (after some time ... ) threw an error: (...) pg_dump: saving dat

Re: [GENERAL] pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists

2015-07-07 Thread pinker
John McKown wrote > ​What version of PostgreSQL? What OS? What was the command line? *OS*: Linux OptiPlex-760 3.8.0-19-generic #29-Ubuntu SMP Wed Apr 17 18:16:28 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux Linux Mint 15 Olivia *pg_dump*: pg_dump (PostgreSQL) 9.4.1 *command line:* opt/PostgreSQL/9.4/

Re: [GENERAL] pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists

2015-07-07 Thread pinker
Adrian Klaver-4 wrote > So this was not the actual command you ran as I see no path specification. > > Some notes: > > 1) plain format is the default so it does not need to specified. > > 2) Either --inserts or --column-inserts not both. FYI this really slows > down the restore process. > > 3)

Re: [GENERAL] pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists

2015-07-07 Thread pinker
Adrian Klaver-4 wrote > So how did you get the wrong file name then, the pgAdmin backup dialog > has a file selector? > > Or did you cut and paste into the file field? I've cut, pasted and changed parameters. -- View this message in context: http://postgresql.nabble.com/pg-dump-PostgreSQL-

Re: [GENERAL] pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists

2015-07-07 Thread pinker
Adrian Klaver-4 wrote > So how did you get the wrong file name then, the pgAdmin backup dialog > has a file selector? > > Or did you cut and paste into the file field? Anyway, shouldn't postgresql first check if dump can be saved? and then start doing it? -- View this message in context: htt

Re: [GENERAL] pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists

2015-07-07 Thread pinker
Adrian Klaver-4 wrote > I was talking about when you ran the backup from pgAdmin. The backup > dialog has a file selector button to select the path/file you want to > back up to. So I was wondering how you got an incorrect path in the > first place? Then it occurred to me you might not have hav

Re: [GENERAL] pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists

2015-07-09 Thread pinker
Adrian Klaver-4 wrote > That is sort of dangerous:) As an example, do you really want --inserts > or --column-inserts, they really slow down a restore. I know, but this time I need it more for versioning/demo version prepariation so performence isn't important at all, what I care about is only dd

Re: [GENERAL] pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists

2015-07-09 Thread pinker
Tom Lane-2 wrote > $ time pg_dump -f /z/q regression > pg_dump: [archiver] could not open output file "/z/q": No such file or > directory > > real0m1.164s > user0m0.062s > sys 0m0.034s > > However, I don't see that with any of the non-plain-text output formats: In my case this is: p

[GENERAL] SELECT clause without parameters

2015-08-17 Thread pinker
I would like to ask what's the reason of change SELECT behaviour. In distributions below 9.4 SELECT without any parameters caused a syntax error and now gives empty set. Was it made for some bigger aim ? :) for instance 8.4: postgres=# select version();

[GENERAL] Shouldn't "WHEN (OLD.* IS DISTINCT FROM NEW.*)" clause be independent from data type?

2015-09-17 Thread pinker
I've tried to write audit trigger which fires only when data changed, so I used "WHEN (OLD.* IS DISTINCT FROM NEW.*)" clause as described in documentation . Should this clause be independent from data type? because an error occurs

[GENERAL] Re: Shouldn't "WHEN (OLD.* IS DISTINCT FROM NEW.*)" clause be independent from data type?

2015-09-17 Thread pinker
Yes I will try to do something else like casting, hashing or using another operator but in documentation this example stands as a model audit trigger without any warnings... -- View this message in context: http://postgresql.nabble.com/Shouldn-t-WHEN-OLD-IS-DISTINCT-FROM-NEW-clause-be-independ

[GENERAL] Escaping text / hstore

2015-10-20 Thread pinker
Maybe somebody have an idea how to escape text string for use in hstore column? I have tried $$ and quote_literal in audit trigger function, but still db won't let me pass values with // or ' to the hstore...INSERT FROM trigger function: EXECUTE 'INSERT INTO history.' || TG_TABLE_SCHEMA || '_'

Re: [GENERAL] Escaping text / hstore

2015-10-20 Thread pinker
Adrian Klaver-4 wrote > Here is my very similar function: > > CREATE OR REPLACE FUNCTION utility.archive_del_record() RETURNS trigger AS > $BODY$ > DECLARE > tbl_name text := TG_TABLE_NAME || '_delete' ; > archive_row hstore := hstore(OLD.*); > user_name text := session_user; > BEGI

[GENERAL] Sequential vs. random values - number of pages in B-tree

2016-08-18 Thread pinker
Hi! After doing a quick test: with sequential values: create table t01 (id bigint); create index i01 on t01(id); insert into t01 SELECT s from generate_series(1,1000) as s; and random values: create table t02 (id bigint); create index i02 on t02(id); insert into t02 SELECT random()*100 from ge

Re: [GENERAL] Sequential vs. random values - number of pages in B-tree

2016-08-18 Thread pinker
W dniu 2016-08-18 14:00:31 użytkownik Ilya Kazakevich napisał: > Hi, > > >What's the reason that postgres needs more index pages to store random > >data > >than sequential ones? > > I assume that is because B-Tree is self-balanced tree, so it needs to be > rebalanced after each insertion. >

Re: [GENERAL] Sequential vs. random values - number of pages in B-tree

2016-08-18 Thread pinker
W dniu 2016-08-18 14:19:25 użytkownik Ilya Kazakevich napisał: > >Thank you. So if that is the reason changing the fillfactor parameter should > >help? > > Fillfactor is not about rebalancing, but about page split. If you have many > insertions you may decrease fillfactor to minimize page sp

Re: [GENERAL] Sequential vs. random values - number of pages in B-tree

2016-08-23 Thread pinker
Francisco Olarte wrote > It's already been told that btrees work that way, if you find itstrange > read a bit about them, this is completely normal, but ... I am just surprised by the order of magnitude in the difference though. 2 and 27 minutes that's the huge difference...I did another, simplifi

[GENERAL] Re: performance problems with bulk inserts/updates on tsrange with gist-based exclude constrains

2016-09-21 Thread pinker
Jeff Janes wrote > Try swapping the order of the columns in the exclude constraint. You want > the more selective criterion to appear first in the index/constraint. > Presumably "key with =" is the most selective, especially if many of your > periods are unbounded. I would not be so sure with tha

[GENERAL] pg_am access in simple transaction?

2016-11-25 Thread pinker
Hi, I'm doing simple tests with lsof on data catalog with bash script: #!/bin/bash for i in {0..2000} do psql -U postgres -c '*BEGIN; select pg_sleep(30); COMMIT*'& done and i measure number of open files and what files are affected by specific command. Lsof has shown me that the only fil

Re: [GENERAL] pg_am access in simple transaction?

2016-11-25 Thread pinker
Adrian Klaver-4 wrote > I can not replicate using 50 clients instead of 2000. I suspect either > has to do with the extreme number of clients or it is an artifact of > from some other process. And I have replicated it with 50 clients as well... lsof output: 51 data/base/13328/2601 command: wat

[GENERAL] Strange activity of prepared queries

2016-12-02 Thread pinker
Hi! I have pgBadger report with strange data about prepared queries I cannot interpret by myself. If anybody could help me with interpretation, would be greatly appreciated. In first half of the day pgBadger shows there is no prepare statements and all queries are binded - how is that possible if a

[GENERAL] Database block lifecycle

2014-08-12 Thread pinker
Yesterday I had an interesting discussion with my colleague about shared buffers size for our new server. This machine (is dedicated for db) has got 512GB of RAM and database size is about 80GB, so he assumes that db will never have to read from disk, so there is no point to adjust read ahead setti

Re: [GENERAL] Database block lifecycle

2014-08-12 Thread pinker
yes, I know the count is quite high. It is the max value we've estimated, but probably on average day it will be 100-200, and yes we use pgpool. Am 13.08.2014 00:09, schrieb John R Pierce: On 8/12/2014 2:41 PM, pinker wrote: btw. 512MB if we assume up to 600 connection is a reasonable

Re: [GENERAL] Database block lifecycle

2014-08-12 Thread pinker
amount of RAM available? Am 13.08.2014 00:39, schrieb John R Pierce: On 8/12/2014 3:29 PM, pinker wrote: yes, I know the count is quite high. It is the max value we've estimated, but probably on average day it will be 100-200, and yes we use pgpool. if you're using a pooler, t

Re: [GENERAL] Database block lifecycle

2014-08-13 Thread pinker
Jeff Janes wrote > On Tuesday, August 12, 2014, pinker < > pinker@ > > wrote: > >> Do you ever plan on restarting this server? Doing maintenance? Applying >> security patches? > > Sure, I assumed when db is up and running, of course after first read from

[GENERAL] Optimal checkpoint_setting

2014-10-09 Thread pinker
Hello All, I have a brand new machine to tune: x3750 M4, 4xE5-4640v2 2.2GHz; 512GBRAM (32x16GB), 4x300GB SAS + SSD (Easy Tier) in RAID 10 What's particularly important now is to choose optimal configuration for write operations. We have discussion about checkpoint_segments and checkpoint_timeout p

Re: [GENERAL] Optimal checkpoint_setting

2014-10-14 Thread pinker
Thank you for your answer, but I don't have access to this server any more and have to just interpret and pick some parameters from test results, but your advice about tar trick is something I'm gonna try for sure in next test. So, because I don't have any chance to do some more tests, should I ch

[GENERAL] Mistake in documentation? ANALYZE on partitioned tables

2014-12-03 Thread pinker
Hi! As I have read in documentation that "If you are using manual VACUUM or ANALYZE commands, don't forget that you need to run them on each partition individually. A command like: ANALYZE measurement; will only process the master

Re: [GENERAL] Merge rows based on Levenshtein distance

2014-12-03 Thread pinker
There is nice extension in postgres: fuzzystrmatch I have used to calculate the distance. From documetation: SELECT levenshtein_less_equal('extensive', 'exhaustive',2); You can use it then with your group by query. -- View this

[GENERAL] Check if LDAP Authentication is used for user

2014-12-23 Thread pinker
Is there any way to check from SQL or psql level if for roles LDAP Authentication is used? There is no information in pg_roles or pg_shadow. -- View this message in context: http://postgresql.nabble.com/Check-if-LDAP-Authentication-is-used-for-user-tp5831817.html Sent from the PostgreSQL - gene

Re: [GENERAL] Check if LDAP Authentication is used for user

2014-12-23 Thread pinker
thank you for the answer. -- View this message in context: http://postgresql.nabble.com/Check-if-LDAP-Authentication-is-used-for-user-tp5831817p5831847.html Sent from the PostgreSQL - general mailing list archive at Nabble.com. -- Sent via pgsql-general mailing list (pgsql-general@postgresq

[GENERAL] infinite recursion detected in rules for relation

2015-02-12 Thread pinker
I wanted to set a rule: CREATE RULE "_RETURN" ASON SELECT * from backend.testDO INSTEAD SELECT * FROM backend.test WHERE who='Me'; When I'm trying to do anything on the table I get following error: ERROR: infinite recursion detected in rules for relation "backend.test"** Błą

Re: [GENERAL] infinite recursion detected in rules for relation

2015-02-13 Thread pinker
hmm I wanted to show only rows that was inserted today, so instead of who='me' wanted to filter for instance where timestamp_column=CURRENT_DATE. Yes, a view would be a solution but I thouhgt that's the case rules were made for? Isn't it? -- View this message in context: http://postgresql.nabb

Re: [GENERAL] infinite recursion detected in rules for relation

2015-02-13 Thread pinker
Ok, but in this particular case I don't see any caveats and think that could be classic case for rule to be used. If it is "almost never the correct solution" why rules still exists at all? -- View this message in context: http://postgresql.nabble.com/infinite-recursion-detected-in-rules-for-re

[GENERAL] Using ctid in delete statement

2017-02-15 Thread pinker
Hi, is it safe to use ctid in following query? : DELETE FROM table_name WHERE ctid = any ( array ( select tn.ctid from table_name tn JOIN items i on tn.itemid=i.itemid WHERE tn.clock < extract ( epoch FROM now() - i.history * interval '10 day')::int + 6 limit 100)); Could I be sure that ctid will

Re: [GENERAL] Using ctid in delete statement

2017-02-15 Thread pinker
thank you for the answer -- View this message in context: http://postgresql.nabble.com/Using-ctid-in-delete-statement-tp5944434p591.html Sent from the PostgreSQL - general mailing list archive at Nabble.com. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make ch

Re: [GENERAL] Using ctid in delete statement

2017-02-16 Thread pinker
Thank you Tom for clarification. Does it mean that FOR UPDATE clause works with other operations as well? i.e. TRUNCATE, DELETE? -- View this message in context: http://postgresql.nabble.com/Using-ctid-in-delete-statement-tp5944434p5944658.html Sent from the PostgreSQL - general mailing list ar

Re: [GENERAL] Using ctid in delete statement

2017-02-16 Thread pinker
Adrian Klaver-4 wrote > https://www.postgresql.org/docs/9.6/static/sql-truncate.html > > https://www.postgresql.org/docs/9.6/static/sql-delete.html There is nothing about FOR UPDATE clause on those pages... -- View this message in context: http://postgresql.nabble.com/Using-ctid-in-delete-st

Re: [GENERAL] Using ctid in delete statement

2017-02-16 Thread pinker
Adrian Klaver-4 wrote > Exactly, they do not have it whereas: > > https://www.postgresql.org/docs/9.6/static/sql-select.html#SQL-FOR-UPDATE-SHARE Still not much. The documentation could be more verbose on this topic. I can only presume that since there is an example with select: SELECT * FROM (SE

[GENERAL] Index loading methods

2017-03-27 Thread pinker
If PostgreSQL decides to use an index, does he every time load the whole B-tree into memory? or maybe loads only specific subtree or some chunks of index? -- View this message in context: http://www.postgresql-archive.org/Index-loading-methods-tp5952220.html Sent from the PostgreSQL - general m

[GENERAL] Huge Pages - setting the right value

2017-03-30 Thread pinker
Hi, I'm currently testing performance with and without huge pages. Documentation says that in order to estimate the number of huge pages needed one should check the postmaster's VmPeak value. I wonder if it's only postmaster memory usage what's matters? Or I could get better estimation from the mos

Re: [GENERAL] Huge Pages - setting the right value

2017-03-30 Thread pinker
W dniu 2017-03-30 11:45:55 użytkownik pinker napisał: > Hi, > I'm currently testing performance with and without huge pages. Documentation > says that in order to estimate the number of huge pages needed one should > check the postmaster's VmPeak value. I wonder if it&#x

[GENERAL] Archiving data to another server using copy, psql with pipe

2017-04-05 Thread pinker
Hi, I'm trying to write an archive manager which will be first copying data from tables with where clause and then, after successful load into second server - delete them. The simplest (and probably fastest) solution I came up with is to use copy: psql -h localhost postgres -c "copy (SELECT * FROM

Re: [GENERAL] Archiving data to another server using copy, psql with pipe

2017-04-06 Thread pinker
W dniu 2017-04-06 13:24:16 użytkownik Moreno Andreo napisał: > Il 05/04/2017 23:26, pinker ha scritto: > > Hi, > > I'm trying to write an archive manager which will be first copying data from > > tables with where clause and then, after successful load into secon

Re: [GENERAL] Archiving data to another server using copy, psql with pipe

2017-04-06 Thread pinker
W dniu 2017-04-06 14:28:04 użytkownik Moreno Andreo napisał: > Il 06/04/2017 13:58, pinker ha scritto: > > > > W dniu 2017-04-06 13:24:16 użytkownik Moreno Andreo > > napisał: > >> Il 05/04/2017 23:26, pinker ha scritto: > >>> Hi, > >>>

Re: [GENERAL] Block size recommendation for Red Hat Linux 7.2

2017-04-24 Thread pinker
I've seen very big differences with huge_pages set to on, especially in context of CPU usage on multiple socket servers. You could play as well with storage options, for instance inode size and check if there is any advantage for your db from inlining, which is supported by xfs. You can find more

Re: [GENERAL] Query which "never" terminates

2017-04-27 Thread pinker
Try to change from_collapse_limit values at first, for instance run SET from_collapse_limit = 1; and then your query in the same session. have you ran analyze lately? -- View this message in context: http://www.postgresql-archive.org/Query-which-never-terminates-tp5958734p5958736.html Sent fr

Re: [GENERAL] Huge Pages - setting the right value

2017-06-11 Thread pinker
We are experiencing some performance issues because of high CPU load. So I would like to ask one more time. The exact question is: Does PostgreSQL can use huge pages for processes or only for shared buffers? (Does it make any sense to set the number of huge pages above the shared_buffers?) Any help

Re: [GENERAL] Huge Pages - setting the right value

2017-06-11 Thread pinker
Andrew Kerber wrote > I can't give you an absolutely authoritative answer, but because of the > way hugepages are implemented and allocated, I can't think how they could > be used for other processes. Linux hugepages are either 2m or 1g, far too > large for any likely processes to require. They ca

Re: [GENERAL] Huge Pages - setting the right value

2017-06-12 Thread pinker
standard hugepages, transparent are disabled. They were set exactly following the procedure from postgres documentation. -- View this message in context: http://www.postgresql-archive.org/Huge-Pages-setting-the-right-value-tp5952972p5966064.html Sent from the PostgreSQL - general mailing list a

Re: [GENERAL] How to drop column from interrelated views

2017-07-09 Thread pinker
I've got some functions on my own :) If you prefer to do it from SQL level I've got two functions prepared which first secure then restore all missing views definitions. -- secure all views DO $$ BEGIN --drop schema migration cascade CREATE SCHEMA migration; CREATE TABLE migration.views AS

Re: [GENERAL] Audit based on role

2017-08-08 Thread pinker
Hi, I personally don't like the solution from https://wiki.postgresql.org/wiki/Audit_trigger and here are the reasons why: * it produces a large table where all the changes, from all tables and schemas go - audit.logged_actions. So when you would like to read it to check anything it will be super s

[GENERAL] Checkpoint write time - anything unusual?

2017-10-02 Thread pinker
I've just run pgBadger on my pg logs and wonder if those checkpoint statistics is something I should worry about or not? The highest write time is about 47 minutes but I'm not sure if that's checkpoint_completion_target*checkpoint_target value or real time between sending the command to write and g

Re: [GENERAL] delete a file everytime pg server starts/crashes

2017-10-06 Thread pinker
Look at inotify: https://github.com/rvoicilas/inotify-tools You can check for instance if postmaster.pid exists. -- Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subsc

Re: [GENERAL] delete a file everytime pg server starts/crashes

2017-10-06 Thread pinker
Actually if the name of the file doesn't matter you could put it into $PGDATA/pg_stat and name it global.stat. When postgres stops (clean or because of failure), replaces the file with his own. So your content will be erased. I'm not sure it's completely safe but works in simple test. -- Sent fr

[GENERAL] core system is getting unresponsive because over 300 cpu load

2017-10-10 Thread pinker
Hi to all! We've got problem with a very serious repetitive incident on our core system. Namely, cpu load spikes to 300-400 and the whole db becomes unresponsive. From db point of view nothing special is happening, memory looks fine, disks io's are ok and the only problem is huge cpu load. Kernel

Re: [GENERAL] core system is getting unresponsive because over 300 cpu load

2017-10-10 Thread pinker
Thank you Scott, we are planning to do it today. But are you sure it will help in this case? -- Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.

Re: [GENERAL] core system is getting unresponsive because over 300 cpu load

2017-10-10 Thread pinker
Victor Yegorov wrote > Can you provide output of `iostat -myx 10` at the “peak” moments, please? sure, please find it here: https://pastebin.com/f2Pv6hDL Victor Yegorov wrote > Also, it'd be good to look in more detailed bgwriter/checkpointer stats. > You can find more details in this post: http

Re: [GENERAL] core system is getting unresponsive because over 300 cpu load

2017-10-10 Thread pinker
Scott Marlowe-2 wrote > Ouch, unless I'm reading that wrong, your IO subsystem seems to be REALLY > slow. it's a huge array where a lot is happening, for instance data snapshots :/ the lun on which is this db is dm-7. I'm a DBA with null knowledge about arrays so any advice will be much appreciate

Re: [GENERAL] core system is getting unresponsive because over 300 cpu load

2017-10-10 Thread pinker
Tomas Vondra-4 wrote > What is "CPU load"? Perhaps you mean "load average"? Yes, I wasn't exact: I mean system cpu usage, it can be seen here - it's the graph from yesterday's failure (after 6p.m.): So as one can see connections spikes foll

Re: [GENERAL] core system is getting unresponsive because over 300 cpu load

2017-10-10 Thread pinker
Victor Yegorov wrote > Looks like `sdg` and `sdm` are the ones used most. > Can you describe what's on those devices? Do you have WAL and DB sitting > together? > Where DB log files are stored? it's multipath with the same LUN for PGDATA and pg_log, but separate one for xlogs and archives. mpatha

Re: [GENERAL] core system is getting unresponsive because over 300 cpu load

2017-10-10 Thread pinker
Tomas Vondra-4 wrote > I'm probably a bit dumb (after all, it's 1AM over here), but can you > explain the CPU chart? I'd understand percentages (say, 75% CPU used) > but what do the seconds / fractions mean? E.g. when the system time > reaches 5 seconds, what does that mean? hehe, no you've just s

Re: [GENERAL] core system is getting unresponsive because over 300 cpu load

2017-10-10 Thread pinker
Andres Freund wrote > Others mentioned already that that's worth improving. Yes, we are just setting up pgbouncer Andres Freund wrote > Some versions of this kernel have had serious problems with transparent > hugepages. I'd try turning that off. I think it defaults to off even in > that version

[GENERAL] Block duplications in a shared buffers

2017-11-07 Thread pinker
Hi All, I was analysing shared buffers content and noticed that exactly the same disk block appears there many times with different or the same usagecount. What's the cause of that? It's because of transactions? SELECT count(*), relfilenode, relblocknumber, array_agg(usagecount) usagecoun

Re: [GENERAL] Block duplications in a shared buffers

2017-11-07 Thread pinker
Tom Lane-2 wrote > Postgres would be completely broken if that were true, because > modifications made to one copy would fail to propagate to other copies. > I don't know where your data came from, but it can't be an accurate > representation of the instantaneous state of the buffer cache. > > ...

Re: [GENERAL] Block duplications in a shared buffers

2017-11-07 Thread pinker
Thank you Tom, you were right. I needed to group by all the columns: reldatabase, reltablespace, relforknumber too. Now all of them are unique. Thank you for clearing this out :) -- Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html -- Sent via pgsql-general mailing