Re: [GENERAL] Two-way encryption

2014-07-01 Thread Jacob Bunk Nielsen
Patrick Simcoe writes: > Does anyone have a technique or recommendation for two-way encryption > which somehow obfuscates the decrypt key so that it isn't easily > retrievable from the database or the application source code? We've > already considered (a) letting users hold the decrypt key and (

[GENERAL] How can I replace the year of the created_at column with the current year dynamically ?

2014-07-01 Thread Arup Rakshit
Here is my try : staging::=> select  to_char(created_at,'DD/MM') || '/' || to_char(now(),'') as when from users;     when      24/02/2014  28/02/2014  02/03/2014  01/03/2014  04/03/2014  02/03/2014  06/03/2014  07/05/2014  02/06/2014  06/06/2014  20/02/2014  20/02/2014  20/02/201

Re: [GENERAL] Validating User Login Within Postgres

2014-07-01 Thread Sim Zacks
On 07/01/2014 06:03 PM, Rich Shepard wrote: On Tue, 1 Jul 2014, hubert depesz lubaczewski wrote: That depends. For example - for system that will have 5 users, and requires strict security policies - it would make sense. On the

Re: [GENERAL] pl/perl and recent perl versions - failing to load internal modules

2014-07-01 Thread Alex Hunsaker
On Tue, Jul 1, 2014 at 7:31 PM, Alex Hunsaker wrote: > On Tue, Jul 1, 2014 at 6:39 PM, Toby Corkindale > wrote: >> Hi Alex, >> However the following code fails: >> => do $$ "\N{U+263A}" =~ /[[:punct:]]/$$ language plperl; >> ERROR: Unable to load utf8.pm into plperl at line 1. >> BEGIN failed-

Re: [GENERAL] pl/perl and recent perl versions - failing to load internal modules

2014-07-01 Thread Alex Hunsaker
On Tue, Jul 1, 2014 at 6:39 PM, Toby Corkindale wrote: > Hi Alex, > your example (chr(0x100) =~ /\\xa9/) works on my instance (pg 9.3.4, plperl > 5.18) > However the following code fails: > => do $$ "\N{U+263A}" =~ /[[:punct:]]/$$ language plperl; > ERROR: Unable to load utf8.pm into plperl at

Re: [GENERAL] Two-way encryption

2014-07-01 Thread Toby Corkindale
- Original Message - > From: "Patrick Simcoe" > To: pgsql-general@postgresql.org > Sent: Wednesday, 2 July, 2014 1:42:04 AM > Subject: [GENERAL] Two-way encryption > > I have a question regarding two-way encryption data for specific columns. > > Does anyone have a technique or recommenda

Re: [GENERAL] what specifically does vacuum have to scan / why does it need to rescan the same indexes many, many times

2014-07-01 Thread Jeff Janes
On Tuesday, July 1, 2014, john gale wrote: > > What does vacuum have to scan to be able to reclaim space, and how many > times does it need to scan to finalize ? > > More specifically, my VACUUM VERBOSE is taking a long time and > seems to be rescanning the same indexes / fields multiple times w

[GENERAL] what specifically does vacuum have to scan / why does it need to rescan the same indexes many, many times

2014-07-01 Thread john gale
What does vacuum have to scan to be able to reclaim space, and how many times does it need to scan to finalize ? More specifically, my VACUUM VERBOSE is taking a long time and seems to be rescanning the same indexes / fields multiple times without finishing. db=# vacuum verbose testruns; INFO

Re: [GENERAL] pl/perl and recent perl versions - failing to load internal modules

2014-07-01 Thread Toby Corkindale
Hi Alex, your example (chr(0x100) =~ /\\xa9/) works on my instance (pg 9.3.4, plperl 5.18) However the following code fails: => do $$ "\N{U+263A}" =~ /[[:punct:]]/$$ language plperl; ERROR: Unable to load utf8.pm into plperl at line 1. BEGIN failed--compilation aborted. CONTEXT: PL/Perl anonymo

Re: [GENERAL] pl/perl and recent perl versions - failing to load internal modules

2014-07-01 Thread Toby Corkindale
Hi Alex, => do $$ "\N{U+263A}" =~ /[[:punct:]]/ $$ language plperl; ERROR: Unable to load utf8.pm into plperl at line 1. BEGIN failed--compilation aborted. CONTEXT: PL/Perl anonymous code block - Original Message - > From: "Alex Hunsaker" > To: "Toby Corkindale" > Cc: "pgsql-general"

Re: [GENERAL] hstore to json and back again

2014-07-01 Thread Paul Jungwirth
> I'm trying to migrate an existing hstore column to json in Postgres > 9.3, and I'd like to be able to run the script in reverse. To answer my own question, this partially solves the problem for me (foo.datahash_new has json, foo.datahash_old has hstore): connection.select_rows(<<-EOQ).each

Re: [GENERAL] pl/perl and recent perl versions - failing to load internal modules

2014-07-01 Thread Alex Hunsaker
On Mon, Jun 30, 2014 at 7:47 PM, Toby Corkindale wrote: > Hi, > I've been trying out PostgreSQL 9.3 with pl/perl built against Ubuntu 14.04 > LTS' Perl 5.18 > (Sourced from apt.postgresql.org) > > Maybe I'm doing something wrong, but it appears that plperl has become > completely useless, as it

[GENERAL] hstore to json and back again

2014-07-01 Thread Paul Jungwirth
Hello, I'm trying to migrate an existing hstore column to json in Postgres 9.3, and I'd like to be able to run the script in reverse. I know not all JSON can turn back into hstore, but since this is coming from an old hstore column, I know the structure is flat (no nesting), and that all values ar

Re: [GENERAL] lock contention, need profiling idea

2014-07-01 Thread Merlin Moncure
On Tue, Jul 1, 2014 at 2:28 PM, Jeff Janes wrote: >> You should have a look at pg_stat_activity, pg_prepared_xacts and pg_locks >> to get more information about the transactions running and the locks being >> taken. > > In 9.4, the log message will also include info on the blocking > process, not

Re: [GENERAL] how to create a role with no privileges?

2014-07-01 Thread Kynn Jones
On Tue, Jul 1, 2014 at 1:28 PM, David G Johnston wrote: > The first rule regarding PostgreSQL permissions is that everything is > forbidden unless allowed - via GRANT. REVOKE simply undoes whatever has > been granted; it does not put up a block to prevent inheritance of granted > permissions. >

Re: [GENERAL] lock contention, need profiling idea

2014-07-01 Thread Jeff Janes
On Mon, Jun 30, 2014 at 5:06 PM, Michael Paquier wrote: > On Tue, Jul 1, 2014 at 7:36 AM, AI Rumman wrote: >> >> I see lots of similar log message at a certain time in a day on Postgresql >> 9,.1: >> >> LOG: process 18855 still waiting for ShareLock on transaction 2856146023 >> after 1001.209 ms

Re: [GENERAL] Best way to list a role s owned objects?

2014-07-01 Thread Tom Lane
Jerry Sievers writes: > Felipe Gasper writes: >> Every database on the cluster, individually, then? Is there no way to >> query all databases at once? >> I mean, *something* under the hood must be doing this because DROP >> ROLE bugs out if the role owns anything in any DB. > That is made possib

Re: [GENERAL] Best way to list a roles owned objects?

2014-07-01 Thread Jerry Sievers
Jerry Sievers writes: > Felipe Gasper writes: > >> On 7/1/14 1:13 PM, John R Pierce wrote: >> >>> On 7/1/2014 11:08 AM, Felipe Gasper wrote: What is the best way to list a role’s owned objects in any database? >>> >>> query pg_class in each database ? >>> >> >> Every database on the clu

Re: [GENERAL] lock contention, need profiling idea

2014-07-01 Thread Merlin Moncure
On Mon, Jun 30, 2014 at 5:36 PM, AI Rumman wrote: > I see lots of similar log message at a certain time in a day on Postgresql > 9,.1: > > LOG: process 18855 still waiting for ShareLock on transaction 2856146023 > after 1001.209 ms > STATEMENT: UPDATE table1 SET time = $1 WHERE id = $2 > > The t

Re: [GENERAL] Best way to list a roles owned objects?

2014-07-01 Thread Jerry Sievers
Felipe Gasper writes: > On 7/1/14 1:13 PM, John R Pierce wrote: > >> On 7/1/2014 11:08 AM, Felipe Gasper wrote: >>> What is the best way to list a role’s owned objects in any database? >> >> query pg_class in each database ? >> > > Every database on the cluster, individually, then? Is there n

Re: [GENERAL] Best way to list a role’s owned objects?

2014-07-01 Thread Felipe Gasper
On 7/1/14 1:13 PM, John R Pierce wrote: On 7/1/2014 11:08 AM, Felipe Gasper wrote: What is the best way to list a role’s owned objects in any database? query pg_class in each database ? Every database on the cluster, individually, then? Is there no way to query all databases at once?

Re: [GENERAL] Best way to list a role’s owned objects?

2014-07-01 Thread John R Pierce
On 7/1/2014 11:08 AM, Felipe Gasper wrote: What is the best way to list a role’s owned objects in any database? query pg_class in each database ? -- john r pierce 37N 122W somewhere on the middle of the left coast -- Sent via pgsql-general mailing li

[GENERAL] Best way to list a role’s owned objects?

2014-07-01 Thread Felipe Gasper
Hi all, What is the best way to list a role’s owned objects in any database? My problem is that I need to be able to “FORCE DROP ROLE” by taking any objects that that role owns, in any database, and reassigning them to the “postgres” user. Even if REASSIGN OWNED BY worked for this

Re: [GENERAL] how to create a role with no privileges?

2014-07-01 Thread David G Johnston
Kynn Jones wrote > On Tue, Jul 1, 2014 at 12:36 PM, Francisco Olarte < > folarte@ > > > wrote: > >> Without seeing your actual commands, it's difficult to know about the >> schema stuff... >> > > Well, the "actual commands" is what the original question was asking for, > since I really don't kn

Re: [GENERAL] lock contention, need profiling idea

2014-07-01 Thread AI Rumman
There was no CREATE INDEX command running on the host. On Mon, Jun 30, 2014 at 5:06 PM, Michael Paquier wrote: > > > > On Tue, Jul 1, 2014 at 7:36 AM, AI Rumman wrote: > >> I see lots of similar log message at a certain time in a day on >> Postgresql 9,.1: >> >> LOG: process 18855 still waiti

Re: [GENERAL] how to create a role with no privileges?

2014-07-01 Thread Kynn Jones
On Tue, Jul 1, 2014 at 12:36 PM, Francisco Olarte wrote: > Without seeing your actual commands, it's difficult to know about the > schema stuff... > Well, the "actual commands" is what the original question was asking for, since I really don't know how to do any of this (I find the documentation

Re: [GENERAL] how to create a role with no privileges?

2014-07-01 Thread Tom Lane
Kynn Jones writes: > In particular, the minimal role still has access to the shell through \!. > YIKES!!! What exactly do you find to be "yikes" about that? It's a shell under the user's own account on the client machine, ie, exactly like the shell account he invoked psql from. It doesn't reall

Re: [GENERAL] how to create multiple databases running in different dirs

2014-07-01 Thread John R Pierce
On 7/1/2014 7:19 AM, Rémi Cura wrote: If it is so your desire, you could also have multiple server on the same machine (althought on different port). This way each server would have its own repository. which still doesn't answer the question, how would an instance of his program know which

Re: [GENERAL] how to create a role with no privileges?

2014-07-01 Thread Francisco Olarte
On Tue, Jul 1, 2014 at 6:13 PM, Kynn Jones wrote: > Actually, AFAICT, revoking usage on the schemas you listed seems to have no > effect at all on the "minimal role"'s ability to use \l, \d, \dt, etc. > > In particular, the minimal role still has access to the shell through \!. > YIKES!!! Without

Re: [GENERAL] how to create a role with no privileges?

2014-07-01 Thread Kynn Jones
Thanks for your reply. Actually, AFAICT, revoking usage on the schemas you listed seems to have no effect at all on the "minimal role"'s ability to use \l, \d, \dt, etc. In particular, the minimal role still has access to the shell through \!. YIKES!!! kynn On Mon, Jun 30, 2014 at 5:37 PM, J

[GENERAL] Two-way encryption

2014-07-01 Thread Patrick Simcoe
I have a question regarding two-way encryption data for specific columns. Does anyone have a technique or recommendation for two-way encryption which somehow obfuscates the decrypt key so that it isn't easily retrievable from the database or the application source code? We've already considered (a

[GENERAL]

2014-07-01 Thread Patrick Simcoe
I have a question regarding two-way encryption data for specific columns. Does anyone have a technique or recommendation for two-way encryption which somehow obfuscates the decrypt key so that it isn't easily retrievable from the database or the application source code? We've already considered (a

[GENERAL] Help debugging database storage problems

2014-07-01 Thread Jacob Bunk Nielsen
Hi We have a PostgreSQL 9.3.4 running in an LXC container on Debian Wheezy on a Linux 3.10.43 kernel on a Dell R620 server. Data are stored on a XFS file system. We are seeing problems such as: unexpected data beyond EOF in block 2 of relation base/805208133/1238511128 and could not read block

[GENERAL] How does the PostgreSQL partition pruning happens ?

2014-07-01 Thread viraj
Hi , I'm new to postgres and going through it's documentation. I have problem about partition pruning in postgres. In postgres partition is done by child tables. My problem are, 1. when we issue a query to master table , does the query optimizer do partition pruning ? i.e select appropriate chil

Re: [GENERAL] How does the PostgreSQL partition pruning happens ?

2014-07-01 Thread viraj
Got following answer, http://dba.stackexchange.com/questions/69411/postgresql-partition-pruning/69413#69413 -- View this message in context: http://postgresql.1045698.n5.nabble.com/How-does-the-PostgreSQL-partition-pruning-happens-tp5809921p5809923.html Sent from the PostgreSQL - general maili

Re: [GENERAL] Validating User Login Within Postgres

2014-07-01 Thread Rich Shepard
On Tue, 1 Jul 2014, hubert depesz lubaczewski wrote: That depends. For example - for system that will have 5 users, and requires strict security policies - it would make sense. On the other hand, for website, with thousands of users, putting them all as actual roles in Pg doesn't make much sense

Re: [GENERAL] how to create multiple databases running in different dirs

2014-07-01 Thread Rémi Cura
Hey, postgres already takes care of multiple client writting/reading, so you don't really need to be afraid of concurrency (for most of the stuff) If it is so your desire, you could also have multiple server on the same machine (althought on different port). This way each server would have its own

Re: [GENERAL] Validating User Login Within Postgres

2014-07-01 Thread hubert depesz lubaczewski
On Tue, Jul 1, 2014 at 3:58 PM, Rich Shepard wrote: > I'm developing a new application and want to take advantage of postgres > features such as triggers and stored procedures and put as much of the > middleware 'business logic' into the database engine as is practical. > Is it possible, or p

[GENERAL] Validating User Login Within Postgres

2014-07-01 Thread Rich Shepard
I'm developing a new application and want to take advantage of postgres features such as triggers and stored procedures and put as much of the middleware 'business logic' into the database engine as is practical. Is it possible, or practical, to validate a potential user login within the data

[GENERAL] Next steps in debugging database storage problems?

2014-07-01 Thread Jacob Bunk Nielsen
Hi We have a PostgreSQL 9.3.4 running in an LXC container on Debian Wheezy on a Linux 3.10.43 kernel on a Dell R620 server. Data are stored on a XFS file system. We are seeing problems such as: unexpected data beyond EOF in block 2 of relation base/805208133/1238511128 and could not read block

Re: [GENERAL] Migration error " invalid byte sequence for encoding "UTF8": 0xff " from mysql 5.5 to postgresql 9.1

2014-07-01 Thread Kevin Grittner
sunpeng wrote: > load data to postgresql in cmd(encoding is GBK) is WIN8: > psql -h localhost  -d test -U postgres <  dbdata.sql > > I got the error: > ERROR:  invalid byte sequence for encoding "UTF8": 0xff If the encoding is GBK then you will get errors (or incorrect characters) if it is read

[GENERAL] Migration error " invalid byte sequence for encoding "UTF8": 0xff " from mysql 5.5 to postgresql 9.1

2014-07-01 Thread sunpeng
When I do migration from Mysql to PostgreSQL: firstly dump data from mysql in cmd(encoding is GBK) is WIN8: mysqldump -v -nt --complete-insert=TRUE --compatible=postgresql --default-character-set=utf8 --skip-add-locks --compact --no-create-info --skip-quote-names -uroot -p test >dbdata.sql then lo

Re: [GENERAL] collecting employees who completed 5 and 10 years in the current month

2014-07-01 Thread Rebecca Clarke
>From my understanding of what you're saying, you want all the employees that have a 5 year, or 10 year anniversary between today and the start of the current month? If that is the case, then this is what I came up with: select employee_name, to_char(current_date, '')::integer - to_char

Re: [GENERAL] Re: collecting employees who completed 5 and 10 years in the current month

2014-07-01 Thread Rebecca Clarke
Right you are David re my first query. That'll be more appropriate if you want to establish if they're in their 5th year, or 10th year. On Mon, Jun 30, 2014 at 6:08 PM, David G Johnston < david.g.johns...@gmail.com> wrote: > Rebecca Clarke-2 wrote > > create view vw_employee as > >select *